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logisticians  have  Jot  the  current  algorithm  used  to  assess 
the  logistics  s^tus  of  acquisition  programs.  The  study 
also  determin^  the  weights  that  should  be  applied  to  the 
different  -Urt  elements  in  order  to  effectively  measure  the 
logistics  status  of  an  acquisition  program. 

Recommendations  were  made  to  (1)  change  the  weight 
assigned  to  the  ZLS  elements  based  on  program  phase;  (2) 
eliminate  the  Air  Force  Precedence  score  from  the  algorithm; 
(3)  eliminate  the  program  score  from  the  algorithm. 

Recommendations  for  future  research  were  to  (1)  review 
the  issue  of  life  cycle  cost;  (2)  conduct  a  regression 
analysis  on  the  algorithm  to  determine  its  effectiveness; 

(3)  develop  a  method  to  control  the  evaluator  subjectivity, 
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A  PERCEPTION  OF  THE  EFFECTIVENESS  OF  THE  ACQUISITION 
LOGISTICS  ALGORITHM  IN  MEASURING  PROGRAM  STATUS 


I.  Introduction 


Background 

Acquisition  logistics  is  a  management  process  which 
consists  of  the  identification,  assessment,  analysis,  and 
the  resolution  of  logistics  activities  during  the  weapon 
system  acquisition  process  (5:2).  To  assist  the  program 
manager  in  determining  if  he  is  meeting  these  objectives  the 
Air  Force  Acquisition  Logistics  Center  (AFALC)  utilizes  a 
management  information  system  (16). 

Prior  to  implementing  the  management  information 
system,  many  Deputy  Program  Managers  for  Logistics  (DPML) 
expressed  a  concern  that  they  were  not  able  to  adequately 
perform  their  prime  mission  of  ensuring  logistics 
supportability  for  acquisition  programs  because  they  were 
constantly  receiving  requests  for  information  oh  the 
logistics  status  of  the  program  (18;25).  The  purpose  of  the 
management  information  system  was  to  reduce  the  status 
reporting  workload  by  maintaining  a  computer  based  system 
that  had  the  latest  status  of  any  given  acquisition  program. 
This  reduction  in  status  reporting  was  intended  to  enable 
the  DPML  to  have  more  time  to  devote  to  working  logistics 
supportability  issues  (25). 


Research  Issue 


The  Air  Force  Acquisition  Logistics  Center  maintains 
data  on  all  Air  Force  acquisition  programs.  AFALCR  800-2 
tasks  the  Deputy  Program  Manager  for  Logistics  (DPNL)  to 
update  the  status  of  the  ten  Integrated  Logistics  Support 
(ILS)  elements  in  the  Acquisition  Logistics  Management 
Information  System  (ALMIS)  every  30  days  and  whenever  a 
significant  program  event  has  occurred  (1:1).  The  ILS 
elements  are  rated  Red,  Yellow,  or  Green.  A  rating  of  Green 
means  either  an  adequate  get  well  plan  is  available  or  the 
program  is  on  schedule.  A  rating  of  Yellow  is  given  if 
there  is  a  questionable  get  well  plan  or  a  possible  schedule 
slippage.  A  rating  of  Red  indicates  an  inadequate  get  well 
plan,  a  schedule  slippage,  or  a  significant  logistics 
supportability  impact  (14).  Every  month  the  programs  are 
rank  ordered  to  determine  which  programs  most  need 
additional  management  attention.  This  is  accomplished  by 
running  the  sub  program  "Score-Rank".  In  the  "Score-Rank" 
program  a  numeric  weight  is  assigned  to  each  ILS  element 
based  on  the  color  code.  The  program  algorithm  then 
determines  the  rank-order  of  the  programs  (13).  In  1986  the 
number  of  ILS  elements  were  reduced  from  fifteen  to  ten  but 
the  algorithm  wasn't  changed.  Therefore,  the  AFALC  wants  to 
determine  if  the  algorithm  they  are  using  to  measure 
logistics  status  is  perceived  by  the  logistics  community  to 
be  effective  in  identifying  problem  programs  (3). 
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statement  of  the  Problem.  This  research  will  measure 
the  perceptions  that  logisticians  have  of  the  effectiveness 
of  the  current  method  of  evaluating  logistics  program  status 
and  to  determine  what  algorithm  should  be  used  to  measure 
the  logistics  status  of  a  program. 

Specific  Objective 

The  objective  of  this  research  was  to  gather  data  from 
a  random  sample  of  Air  Force  officers  and  Department  of  the 
Air  Force  civilians  assigned  to  logistics  activities  of  AFLC 
and  AFSC  in  order  to  determine  the  perception  of  the  current 
method  of  rank  ordering  acquisition  programs  and  to 
determine  if  different  weights  for  the  ILS  elements  should 
be  used.  To  assist  in  this  effort  the  following  specific 
objectives  were  established: 

1.  Determine  to  what  extent  the  Air  Force  acquisition 
logistics  community  perceives  the  current  algorithm  to  be 
effective  in  measuring  the  status  of  acquisition  programs. 

2.  Determine  what  weights  should  be  assigned  in  the 
algorithm  to  more  effectively  measure  the  status  of 
acquisition  programs. 

Investigative  Questions 

The  specific  objectives  will  be  met  by  analyzing  the 
answers  to  the  following  investigative  questions: 

1.  How  do  logisticians  perceive  the  phases  of  the 
acquisition  process  should  be  weighted? 
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2.  How  do  logisticians  perceive  the  ILS  elements 
should  be  rank-ordered? 

3.  What  is  the  perceived  relationship  between  the 
weights  for  ILS  elements  in  one  phase  compared  to  the  other 
phases? 

4.  Do  acquisition  logisticians  perceive  that  the 
weights  for  the  ILS  elements  are  program  independent? 

Scope  of  the  Study 

This  research  measured  the  perceptions  that 
logisticians  have  of  the  effectiveness  of  the  current  method 
of  evaluating  logistics  program  status  and  assisted  in 
determining  what  weights  should  be  used  for  the  ILS  elements 
to  measure  the  logistics  status  of  acquisition  programs. 

The  collected  data  was  analyzed  to  determine  if  acquisition 
logisticians  have  a  different  perception  of  the 
effectiveness  of  the  current  algorithm  than  support 
logisticians  and  to  determine  the  weights  to  be  used  for  the 
ILS  elements  in  the  algorithm  for  the  different  phases  of 
the  acquisition  process.  The  collected  data  was  analyzed  to 
determine  any  relationships  between  support  and  acquisition 
logisticians  based  on  end  item  management  orientation  and 
the  phase  of  the  acquisition  process. 

Definition  of  Terms 

System  Program  Manager .  Every  Air  Force  acquisition 
program  has  a  program  manager  assigned  in  the  system  program 
office.  The  purpose  of  the  program  manager  is  to  provide 
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the  Air  Force  with  a  single  focal  point  for  the  acquisition 
program.  It  is  the  responsibility  of  the  program  manager  to 
represent  and  protect  the  interests  of  the  Air  Force  in  all 
matters  pertaining  to  the  acquisition  program  (2:2;16:2). 

To  assist  the  system  program  manager  in  the  performance  of 
his  duties  and  responsibilities,  functional  area  specialists 
are  also  assigned  to  the  weapon  system  program  office. 

Deputy  Program  Manager  for  Logistics  ( DPML ) .  The  DPML 
has  the  responsibility  to  ensure  that  logistics 
considerations  are  included  during  the  design  phases  of  the 
acquisition  process.  The  DPML  ensures  that  the  logistics 
issues  which  impact  support ability,  performance,  and 
schedule  are  considered  up  front  in  order  to  reduce  the  life 
cycle  costs  associated  with  the  system. 

Integrated  Logistics  Support  ( ILS) ♦  The  purpose  of  the 
integrated  logistics  support  (ILS)  program  is  to  integrate 
and  correlate  the  technical  and  management  issues  and 
activities  associated  with  an  acquisition  program  (5:1). 

The  integrated  logistics  support  elements  form  the  major 
planning  components  of  the  weapon  system  to  achieve  the 
required  support  capability  at  an  affordable  life  cycle  cost 
(5:l-2;6:2).  Based  on  the  status  of  the  ILS  elements  each 
acquisition  program  is  assigned  a  specific  ranking. 

Acquisition  Logistics  Management  Information  System 
(ALMIS) .  The  Air  Force  Acquisition  Logistics  Center  is 
tasked  with  the  responsibility  of  ensuring  that  the  weapon 
systems  acquired  by  the  Air  Force  will  be  fully  supportable 
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and  able  to  perform  the  mission  they  were  designed  for  in  a 
cost  effective  manner.  Each  program  manager  assesses  his 
program  on  a  monthly  basis  and  inputs  the  resulting  data  in 
the  Acquisition  Logistics  Management  Information  System  data 
base.  Based  on  the  data  in  ALHIS,  each  program  is  given  a 
rank  score  indicating  the  logistics  status  of  the  program 
(13:44-46) . 

ALMIS  is  the  management  information  system  that  is  used 
by  the  Air  Force  Acquisition  Logistics  Center  to  maintain 
information  on  the  status  of  Air  Force  acquisition  programs. 
The  purpose  of  ALHIS  is  to  increase  the  effectiveness  of 
acquisition  logistics  managers  by  identifying  logistics 
problem  areas  in  a  timely  manner.  The  data  base  is  also 
designed  to  reduce  the  logistics  status  reporting  workload 
associated  with  an  acquisition  program. 

Organization  of  the  Thesis 

The  organization  of  the  thesis  will  follow  the  model 
outlined  in  the  AFIT  Style  Guide  for  Theses  and 
Dissertations  (24). 

Chapter  I  of  the  thesis  is  an  introduction  to  the 
research  issue.  It  contains  a  background  of  the  problem, 
the  research  issue  including  the  statement  of  the  problem, 
the  objectives  of  the  research,  the  investigative  questions, 
the  scope  of  the  study,  a  definition  of  terms,  and  the 
organization  of  the  thesis. 

A  review  of  the  pertinent  literature  is  in  chapter  II. 
This  chapter  examines  the  current  and  previous  methods  used 
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to  evaluate  acquisition  logistics  program  status  and  their 
associated  degree  of  success. 

Chapter  III  contains  the  methodology  used  in  gathering 
and  analyzing  the  data.  It  also  provides  information 
relative  to  the  population  of  interest  and  the  sampling 
technique  that  was  used. 

Answers  to  the  investigative  questions  are  in  chapter 
IV  of  the  thesis.  It  also  contains  the  analysis  of  the  data 
and  reports  the  findings. 

Chapter  V  provides  a  summary  of  the  study  and 
recommendations  for  the  direction  that  further  research 
should  take. 
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II.  Literature  Review 


Definitions  and  Relationships 

The  DPNL  ensures  that  the  logistics  issues  which  will 
impact  supportability ,  performance,  and  schedule  are 
properly  considered  on  the  front  end  of  the  system 
acquisition  process  in  order  to  reduce  the  life  cycle  costs 
associated  with  the  system.  The  program  manager  is 
responsible  to  provide  the  leadership  for  logistics 
planning.  This  function  is  achieved  by  establishing  and 
implementing  a  logistics  management  concept  that  is 
responsive  to  the  supportability  and  readiness  objectives  of 
the  weapon  system  (10;16:2). 

The  program  manager  is  assigned  to  the  weapon  system 
acquisition  program  office  by  the  implementing  command.  The 
implementing  command  is  usually. the  Air  Force  Systems 
Command  (AFSC),  although  the  responsibility  for 
implementation  may  be  assigned  to  a  different  organization 
for  a  command  peculiar  acquisition  program  where  educational 
or  technological  expertise  is  a  prerequisite  for  the 
successful  completion  of  the  acquisition.  An  example  of 
such  a  program  would  be  an  upgrade  of  the  military  telephone 
switching  system  for  the  Eastern  United  States  by  using 
satellites.  In  this  example,  the  program  manager  would 
probably  be  assigned  to  the  Air  Force  Space  Command 
(USSPACECOM)  because  of  the  technology  involved  with  space 
systems. 
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In  order  to  assist  the  program  manager  in  the 
performance  of  his  duties  and  responsibilities,  functional 
area  specialists  are  also  assigned  to  the  weapon  system 
program  office.  There  is  no  set  number  of  functional  area 
specialists  assigned  to  a  program  office;  but,  as  a  minimum, 
each  program  office  has  a  functional  expert  assigned  for 
contracting,  engineering,  and  logistics  (16:7). 

The  functional  area  specialist  assigned  to  the  program 
office,  who  is  tasked  with  the  responsibility  to  ensure  that 
logistics  considerations  are  included  during  the  design 
phases  of  the  acquisition  process,  is  the  Deputy  Program 
Manager  for  Logistics  (OPHL).  The  DPHL  is  tasked  with 
ensuring  that  the  weapon  systems,  and  the  related  equipment, 
acquired  by  the  Air  Force  will  be  fully  supportable  and  able 
to  perform  the  mission  they  were  designed  for  in  a  cost 
effective  manner. 

The  DPHL  is  assigned  to  the  system  program  office  by 
the  Air  Force  Acquisition  Logistics  Center  (AFALC)  (4:1-2). 
This  is  usually  accomplished  at  the  beginning  of  the  Concept 
Exploration  Phase  shortly  after  the  milestone  0  decision. 

The  DPHL  is  the  person  who  is  tasked  with  the  responsibility 
to  ensure  that  all  the  logistics  issues  and  considerations 
are  identified  and  included  during  the  design  phases  of  the 
acquisition  process.  This  responsibility  extends  to  the 
logistics  functions  for  the  weapon  system  and  its  related 
equipment.  It  includes  the  ten  Integrated  Logistics  Support 
(ILS)  elements,  the  logistics  budgeting  process,  and  all 
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facets  of  the  support  structure  for  the  complete  system 
including  all  of  the  related  support  equipment  (16:2). 

Although  the  weapon  system  program  manager  has  the 
overall  responsibility  to  establish  and  implement  the  ILS 
program  within  the  system  program  office,  the  DPML  is 
usually  assigned  the  management  responsibility  for  it  by  the 
program  manager.  The  DPHL,  under  the  direction  of  the 
system  program  manager,  integrates  the  ILS  elements  with  the 
design  considerations  of  the  system.  He  is  also  responsible 
for  developing  the  contracting  and  planning  documents 
required  for  the  system  acquisition. 

Integrated  Logistics  Support  ( ILS) .  Integrated 
logistics  support  (ILS)  is  implemented  within  the  Air  Force 
by  means  of  APR  800-8,  Integrated  Logistics  Support  ( ILS) 
Program  (5:1).  This  regulation  defines  the  Air  Force 
program  of  integrated  logistics  support  as  required  by  DODD 
5000.39.  The  purpose  of  the  Air  Force's  integrated 
logistics  support  program  is  to  integrate  and  correlate  all 
of  the  technical  and  management  issues  and  activities 
associated  with  a  weapon  system  acquisition  program  (7:1). 

The  Air  Forces's  integrated  logistics  support  program 
is  used  to  accomplish  the  following  objectives: 

1.  Integrate  support  considerations  into  the 
design  objectives  for  the  system  and  the  related  equipment. 

2.  Increase  sustainability  by  relating  the 
support  requirements  to  readiness  objectives,  to  the  system 
design,  and  to  each  other. 
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3.  Acquire  the  support  needed  for  the  system. 

4.  Provide  required  support  for  the  system  during 
the  operational  phase  consistent  with  the  life  cycle  cost 
objectives  (7;27). 

The  following  ten  Integrated  Logistics  Support  elements 
are  identified  in  DODD  5000.39  and  are  specifically 
addressed  in  the  ILSP  (7:2).  The  integrated  logistics 
support  elements  serve  to  form  the  major  planning  components 
of  the  weapon  system  to  achieve  the  required  support 
capability. 

1.  Maintenance  Planning. 

2.  Manpower  and  Personnel. 

3.  Supply  Support. 

4.  Support  Equipment. 

5.  Technical  Data. 

6.  Training  and  Training  Support. 

7.  Computer  Resources  Support. 

8.  Facilities. 

9.  Packaging,  Handling,  Storage,  and 
Transportation. 

10.  Design  Interface. 

Each  element  is  interrelated  and  interdependent  with 
one  or  more  of  the  other  elements  as  well  as  the  weapon 
systems  design.  The  integration  of  each  element  with  other 
elements  and  the  weapon  system  design  is  achieved  through 
the  LSA  process  (MIL-STD-1388-1A  and  MIL-STD-1388-2A)  and  by 
preparing  subordinate  ILS  elements  plans.  These  plans 
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prescribe  specific  objectives,  strategies,  and  requirements 
for  each  element  in  relationship  to  other  elements,  the 
weapon  system,  and  support  equipment  (  5 : l->2  ;7 :2 } . 

Maintenance  Planning.  The  ILS  element  of 
maintenance  planning  describes  the  maintenance  concept  that 
is  required  to  meet  design  and  readiness  objectives  for  the 
weapon  system  and  the  weapon  system  related  equipment.  The 
maintenance  concept  drives  the  facilities  and  manpower 
requirements  necessary  to  maintain  the  weapon  system  and  the 
weapon  s:,’Stea  related  equipment. 

In  the  long  run,  maintenance  planning  is  one  of  the 
most  significant  factors  that  influences  the  life  cycle 
costs  of  the  weapon  system.  Therefore  it  is  necessary  to 
strike  a  balance  between  maintainabili ty  and  life  cycle 
costs  (2:31-1  to  31-2). 

Manpower  and  Personnel.  The  ILS  element  of 
manpower  and  personnel  is  concerned  with  identifying  the 
proper  personnel  requirements,  with  the  correct  skills  and 
training  levels  that  are  required  to  support  the  weapon 
system  and  the  weapon  system  related  equipment  (2:29-1  to 
29-3). 

Supply  Support.  The  integrated  logistics  support 
element  of  supply  support  consists  of  the  management 
activities  which  are  associated  with  determining  and 
acquiring  the  spares,  both  initial  and  replenishment,  for 
the  weapon  system  and  the  weapon  system  related 
requirements  (2:40-1  to  40-2 ;11). 


Support  Equipment.  The  integrated  logistics 
support  element  of  support  equipment  is  concerned  with 
identifying  and  acquiring  all  the  equipment  needed  to 
support  the  operation  and  maintenance  of  the  weapon  system. 
This  includes  the  ground  handling  and  maintenance  equipment, 
test  equipment,  and  automatic  test  equipment  (21). 

Technical  Data.  The  integrated  logistics  support 
element  of  technical  data  is  concerned  with  the 
identification  and  acquisition  of  technical  orders  and 
engineering  data.  Technical  data  is  all  of  the  information 
of  a  technical  or  engineering  nature  that  is  needed  to 
convert  the  weapon  system  design  specifications  into 
logistics  supportability  considerations  (2:38-1  to 
38: ;5;19) . 

Training  and  Training  Support.  The  integrated 
logistics  support  element  of  training  and  training  support 
is  concerned  with  the  identification  of  the  training 
requirements  needed  to  support  the  weapon  system  and  the 
weapon  system  related  equipment.  In  addition  to  identifying 
the  required  training  it  also  identifies  and  acquires 
training  equipment  and  devices.  It  is  necessary  to  train 
personnel  to  operate  and  maintain  the  weapon  system  in  order 
to  be  able  to  utilize  it.  This  training  consists  of 
equipment  training,  formal  training,  and  on-the-job  training 
(2:39-1  to  39-4). 

Computer  Resources  Support.  The  integrated 
logistics  support  element  of  computer  resources  support  is 
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concerned  with  the  identification  and  acquisition  of 
computer  hardware,  software,  firmware,  and  associated 
computer  documentation  required  to  operate  and  support  the 
weapon  system  and  the  weapon  system  related  equipment.  It 
is  also  concerned  with  the  identification  of  the  technical 
manpower  required  to  support  the  computer  systems  (2:43-2). 

Facilities .  Facilities  are  real  property  assets 
required  to  support  the  weapon  system  consistent  with  the 
operational  and  readiness  requirements.  The  facilities 
integrated  logistics  support  element  ensures  that  the 
required  facilities  are  available  to  the  using  and 
supporting  commands  to  provide  integration  with  the  weapon 
system  support  structure. 

The  integrated  logistics  support  element  of  facilities 
is  concerned  with  the  identification  and  acquisition  of  real 
property  required  to  support  the  weapon  system  and  the 
weapon  system  related  equipment.  It  includes  modifications 
to  currently  existing  facilities  as  well  as  the  construction 
of  new  facilities  required  to  support  the  weapon  system 
(22). 

Packaging,  Handling,  Storage,  and  Transportation . 
The  integrated  logistics  support  element  of  packaging, 
handling,  storage,  and  transportation  is  concerned  with  the 
identification  of  the  requirements,  procedures,  and 
methodologies  needed  to  ensure  that  the  weapon  system  and 
the  weapon  system  related  equipment  are  transportable 
consistent  with  mission  objectives.  This  element  includes 
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storage,  transportation,  and  packaging  requirements  for  the 
system  and  the  system  related  equipment  (20). 

Design  Interface.  The  integrated  logistics 
support  element  of  design  interface  correlates  the  other 
integrated  logistics  support  elements  into  the  design  and 
cost  objectives  for  the  weapon  system  and  the  weapon  system 
related  equipment.  This  is  accomplished  by  relating  the 
system  objectives  with  the  logistics  readiness  objectives. 
Design  interface  is  the  element  that  reviews  the  design 
objectives,  relative  to  logistics,  to  ensure  logistics 
requirements  are  integrated  into  the  weapon  system  and  the 
weapon  system  related  equipment. 

The  design  interface  integrated  logistics  support 
element  helps  to  ensure  that  the  technical  performance  of 
the  weapon  system  is  balanced  with  the  other  integrated 
logistics  support  elements  and  with  the  economic  objectives 
and  constraints  of  the  acquisition  program  (2:30-1). 

Acquisition  Phases.  .  The  acquisition  logistics 
management  of  any  Air  Force  weapon  system  program  is 
dictated  by  DODD  5000.1.  This  directive  outlines  the 
processes  and  procedures  that  govern  the  acquisition  of  any 
new  weapon  system  (9:1).  The  phases  in  the  acquisition 
process  are  as  follows: 

Preconceptual  Phase.  In  the  preconceptual  phase 
the  Statement  of  Operational  Need  (SON)  is  developed.  The 
SON  defines  in  operational  terms  the  specific  requirement  or 
deficiency. 


Concept  Exploration  Phase.  This  phase  is 
concerned  with  the  identification  of  solutions  for  the 
deficiencies  cited  in  the  SON.  During  this  phase,  cost, 
schedule,  support  parameters,  and  performance  parameters  are 
reviewed  and  integrated. 

Demons t rat ion/Valldat ion  Phase.  The  purpose  of 
this  phase  is  to  select  a  system  development  approach  and 
identify  the  risks  associated  with  the  program. 

Full  Scale  Development  Phase.  In  the  full  scale 
development  phase  the  complete  system  is  developed  and 
tested  in  preparation  for  production. 

Production/Deployment  Phase.  The  purpose  of  this 
phase  is  to  produce  and  deploy  the  complete  system  (9;15). 

After  the  military  planners  and  policy  leaders  have 
identified  the  existing  military  threats  and  implemented  the 
recommended  counter  strategies,  the  need  for  a  new  weapon 
system  may  be  identified.  If  the  acquisition  of  a  weapon 
system  is  required  to  counter  the  threat,  the  Air  Force 
begins  the  five  phase  acquisition  process  involved  in  the 
acquisition  of  a  weapon  system  (25). 

Supportability.  If  the  weapon  system  acquisition 
program  is  to  be  successful  it  is  necessary  that  the 
logistics  planning  be  timely.  The  system  program  manager 
and  the  DPHL  must  start  planning  for  logistics 
supportability  from  the  start  of  the  program  and  ensure  that 
the  planning  activities  remain  current  throughout  all  of  the 
phases  of  the  acquisition  process  (6:1). 
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The  formal  planning  process  begins  with  the  inception 
of  the  Program  Management  Plan  (PMP)  during  the  concept 
exploration  phase.  The  program  manager  and  the  DPML's 
involvement  in  planning  for  logistics  considerations  must 
continue  throughout  the  acquisition  or  until  the  management 
of  the  weapon  system  is  turned  over  to  the  supporting 
command  at  program  management  responsibility  transfer 
(PMRT).  The  program  management  plan  outlines  and  details 
the  support  that  is  required  from  the  using  command, 
supporting  command,  implementing  command,  and  any  other 
interested  organizations.  The  deputy  program  manager  for 
logistics,  under  the  direction  of  the  weapon  system  program 
manager,  develops  the  program  management  plan  based  on  the 
acquisition  strategy  (4:1). 

Section  9  of  the  program  management  plan  is  the 
Integrated  Logistics  Support  Plan  (ILSP).  It  is  a 
government  working  document  that  pertains  to  the  logistics 
elements  that  the  government  has  not  contracted  out.  The 
purpose  of  the  ILSP  is  to  determine: 

1.  What  specifically  has  to  be  accomplished. 

2.  Who  is  responsible  to  ensure  that  it  is 
accomplished. 

3.  What  the  schedule  needs  to  be  (5). 

The  integrated  logistics  support  plan  contains  three  main 
parts . 

Part  one  of  the  ILSP  is  a  general  part  that  provides  an 
outline  of  the  system  description  and  program  objectives. 
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Fart  two  of  the  ILSP  contains  the  concepts  and 
strategies  that  will  be  followed  for  the  system.  In 
addition  to  containing  the  acquisition  strategy,  it  also 
contains  the  maintenance,  operational,  and  testing  concepts. 

Part  Three  of  the  ILSP  contains  the  milestone  schedule 
and  resource  reports.  The  ILSP  will  of  necessity  be 
somewhat  vague  and  general  in  nature  during  the  concept 
exploration  phase;  however,  as  the  weapon  system  advances 
the  ILSP  will  define  the  specific  actions  required  for  each 
of  the  ILS  elements  to  ensure  they  are  integrated  with  the 
weapon  system  support  requirements  (16:4). 

Contractual  Issues .  The  logistics  requirements  to 
be  performed  by  contractors  must  be  described  in  full  detail 
in  the  Request  For  Proposal  (RPP)  and  must  be  incorporated 
into  the  Statement  of  Work  (SOW)  and  the  Contract  Data 
Requirements  List  (CDRL)  (27:160-166).  Part  of  this 
logistics  requirement  is  to  task  the  contractor  to  submit 
his  Integrated  Support  Plan  (ESP).  The  contractor's  ISP 
explains  the  approach  the  contractor  plans  to  take  to  ensure 
the  logistics  issues  and  operational  support  requirements 
are  compatible.  The  ISP  will  serve  as  a  method  of  measuring 
the  contractor's  logistics  performance.  The  contractor's 
ISP  must  correlate  with  the  government's  ILSP.  After  the 
contractor's  document  is  approved  by  the  government  it  will 
become  part  of  the  contract. 

The  ISP  format  must  be  the  same  as  the  government's 
ILSP  in  order  for  the  program  office  to  determine  if  the 


contractor  is  performing  as  he  should  relative  to  the 
contractual  logistics  issues. 

Prior  Studies 

In  the  Fall  of  1987,  AFSC/PL  was  concerned  with  the 
ratings  that  were  being  assigned  to  the  individual 
integrated  logistics  support  elements.  This  concern 
centered  around  the  method  that  was  used  to  measure  the 
status  of  the  individual  ILS  elements.  The  evaluation 
method  consisted  of  assigning  a  numeric  value  between  0  and 
9  to  each  ILS  element.  Based  on  the  numeric  value  assigned, 
a  color  code  for  the  element  was  established.  If  the 
numeric  value  was  0  the  color  code  of  Green  was  assigned.  A 
rating  of  1  through  5  indicated  a  rating  of  Yellow.  A  color 
coding  of  Red  was  assigned  to  any  element  with  a  numeric 
value  higher  than  5.  A  rating  of  Green  meant  the  ILS 
element  had  an  adequate  get  well  plan  and  was  on  schedule. 

A  rating  of  Yellow  was  given  if  there  was  a  questionable  get 
well  plan  or  a  possible  schedule  slippage.  A  rating  of  Red 
indicated  an  inadequate  get  well  plan,  a  schedule  slippage, 
or  that  there  was  a  significant  logistics  supportability 
impact  (14).  The  system  then  in  use  was  a  subjective 
evaluation  on  the  part  of  the  program  office  that  created  a 
discontinuity  of  the  ratings  for  the  ILS  elements  from 
program  office  to  program  office.  Because  the  color  coding 
of  the  individual  ILS  elements  was  a  major  portion  of  the 
ILS  assessment  it  was  important  to  develop  a  method  of 
resolving  this  discontinuity.  In  an  attempt  to  provide  some 


structure  to  the  rating  of  the  individual  ILS  elements  a 
policy  letter  was  distributed  by  AFSC/PL  to  each  of  the 
program  offices  in  early  1988  outlining  the  criteria  to  be 
used  in  the  assessment  of  the  individual  ILS  elements. 

The  guidance  that  was  provided  to  the  program  offices 
centered  around  what  color  code  should  be  applied  to  a 
specific  ILS  element  based  on  the  severity  of  the  problems 
associated  with  the  element.  Basically  the  criteria  was  as 
follows: 

A  color  coding  of  Green  should  be  applied  to  an 
ILS  element  that  has  no  problems  or  only  minor  problems  that 
have  solutions  readily  available. 

A  color  coding  of  Yellow  should  be  applied  to  an 
ILS  element  that  has  minor  to  severe  problems,  but  has 
potential  solutions  readily  available. 

A  color  coding  of  Red  should  be  applied  to  an  ILS 
element  that  has  serious  or  potentially  serious  problems 
that  require  higher  headquarters  assistance  to  resolve. 

Hs  Ruth  Klumb  of  the  AFALC  Directorate  of  Systems 
Support,  conducted  a  study  in  June  1987,  on  the  relative 
importance  of  the  individual  ILS  elements  grouped  by  system 
acquisition  phase.  Twenty-eight  DPHLs  from  the  various 
product  divisions  were  surveyed  to  determine  their  views 
concerning  ILS  element  importance  by  phase.  The  study 
indicated  that  there  may  be  a  difference  in  the  importance 
of  the  ILS  elements  in  the  different  Acquisition  phases  but, 
these  differences  were  not  measured  (11;14). 
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In  1979  APALD/XRS  was  tasked  to  determine  how  to 
measure  the  logistics  status  of  Air  Force  acquisition 


programs.  The  algorithm  they  developed  is  used  in  ALHIS  to 
assist  in  the  determination  of  the  status  of  an  acquisition 
program.  The  algorithm  is  as  follows: 

1.  ILS  element  score: 

Green  -  0 
Yellow  •  1  to  5 
Red  >  6  to  9 

The  weight  of  20  times  the  color  coding  weight  for  the 
specific  ILS  element  equals  the  individual  element  score. 

The  sum  of  the  scores  equals  the  weight  assigned  for  the  ILS 
elements . 

2.  Air  Force  Precedence  score: 

The  precedence  score  equals  the  number  80 ,  divided  by 
the  quantity,  1.5  times  the  Force  Activity  Designator  (FAD), 
times  one  third  the  Urgency  of  Need  designator  (UND). 

3.  Program  Phase  score: 

Weights  for  the  acquisition  phases  are  as  follows: 


Production  Deployment  Phase  50 
Pull  Scale  Development  Phase  25 
Demonstration/Validation  Phase  25 
Concept  Exploration  Phase  10 


The  total  program  score  is  arrived  at  by  summing  the  ILS 
Element  score,  the  Air  Force  Precedence  score,  and  the 
Program  Phase  score  (11; 14).  As  an  example,  if  a  program 
had  one  ILS  element  rated  Red  6,  two  elements  rated  as 
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Yellow  3,  and  the  rest  were  rated  as  Green,  it  would  have  an 
ILS  score  as  follows: 


1  X  6  X  20  -  120  - 

2  X  3  X  20  -  120 
7  X  0  X  20  -  0 

240 

If  the  FAD  were  a  2  and  the  UND  were  a  6  it  would  have  a 
precedence  score  of  13. 

_ 80 _ 

(3/2  X  2) (1/3  X  6) 

If  the  program  were  in  the  FSD  phase  it  would  have  a  program 
phase  score  of  25. 

The  total  score  for  the  program  would  be  as  follows: 


ILS  Score 

240 

Precedence  Score 

13 

Program  Phase  Score 

25 

Total  Score 

278 
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HI .  Methodology 


Overview 

This  chapter  describes  the  procedures  used  to  answer 
the  investigative  questions.  These  questions  were  as 
follows: 

1.  How  do  logisticians  perceive  the  phases  of  the 
acquisition  process  should  be  weighted? 

2.  How  do  logisticians  perceive  the  ILS  elements 
should  be  rank-ordered? 

3.  What  is  the  perceived  relationship  between  the 
weights  for  ILS  elements  in  one  phase  compared  to  the  other 
phases? 

4.  Do  acquisition  logisticians  perceive  that  the 
weights  for  the  ILS  elements  are  program  independent? 

Population 

The  population  of  interest  for  this  study  was 
Air  Force  officers  and  civilians  working  as  acquisition 
logisticians  assigned  to  either  the  Air  Force  Acquisition 
Logistics  Center  or  the  AFSC  product  division  program 
offices,  and  their  functional  counterparts  assigned  to  the 
Air  Force  Logistics  Command  (AFLC)  Air  Logistics  Centers  as 
support  logisticians.  Based  on  AFLC  manpower  figures  there 
are  361  civilians  and  164  Air  Force  officers  currently 
assigned  to  these  organizations  with  the  job  series  and 
grades  that  meet  the  population  of  interest. 
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Sample 

Using  a  stratified  random  sample  based  on  current 
program  assignment,  data  was  collected  to  determine  if 
acquisition  logisticians  have  a  different  perception  of  the 
effectiveness  of  the  current  algorithm  than  support 
logisticians.  The  data  was  also  used  to  determine  to  what 
extent  the  current  algorithm  is  perceived  to  be  effective  in 
measuring  logistics  program  status. 

A  survey  was  mailed  to  252  acquisition  logisticians 
assigned  to  AFALC  but  matrixed  to  the  Air  Force  Systems 
Command's  (AFSC)  product  divisions.  The  survey  was  also 
mailed  to  120  support  logisticians  assigned  to  the  AFLC  Air 
Logistics  Centers.  These  organizations  were  chosen  because 
their  item  management  orientation  is  similar  to  the  item 
management  orientation  of  the  AFSC  product  divisions.  The 
names  and  addresses  for  the  respondents  were  provided  by  the 
Air  Force  Acquisition  Logistics  Center  through  the  ALNIS 
data  base.  It  was  assumed  that  the  logistics  users  of  the 
ALNIS  data  base  would  be  the  most  knowledgeable  of  the 
requirements  needed  to  assess  the  status  of  acquisition 
programs. 

Survey  Instrument 

A  mailed  survey  was  developed  and  approved  by  the  Air 
Force  Military  Personnel  Center  (AFHPC),  to  collect 
information  from  a  sample  of  logisticians.  This  approach 
was  used  because  of  the  constraints  of  both  time  and  money. 
This  approach  also  permitted  a  larger  sample  size  to  be 
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surveyed.  The  most  significant  problem  associated  with 
using  a  mailed  survey  is  the  return  rate  of  the  survey  to 
obtain  a  representative  sample.  Therefore,  some  of  the 
techniques  that  Dillman  has  suggested  to  increase  the 
response  rate  were  used.  (12:171-176).  The  cover  letter  for 
the  survey  addressed  not  only  the  need  for  the  respondent's 
help,  but  also  addressed  the  support  the  AFALC  commander  had 
for  the  research.  The  logisticians  were  given  ten  working 
days  from  the  date  of  mailing  to  answer  and  return  the 
survey  in  the  postage  paid,  pre-addressed  envelope.  Five 
working  days  after  the  survey  was  mailed  a  follow  up  letter 
was  sent  thanking  those  who  responded  and  requesting  the 
others  to  complete  and  return  the  survey.  The  survey,  (see 
appendix  A),  contained  demographic  questions,  subject  matter 
perception  questions,  and  structured  open  ended  questions 
designed  to  elicit  data  concerning  the  relative  value  of  the 
ILS  elements  in  the  different  phases  of  the  acquisition 
process.  Question  number  1  through  question  number  6  were 
used  to  gather  demographic  data  to  classify  the  respondents. 
These  questions  were  multiple  choice  questions. 

Question  number  7  through  question  number  22  were 
designed  to  elicit  information  concerning  the  perceptions 
that  logisticians  have  of  the  current  algorithm  and  of  the 
weights  that  are  currently  being  used.  Questions  number  23 
and  24  were  used  to  determine  the  perceptions  that 
logisticians  have  of  the  purpose  of  the  management 
information  system.  A  seven  point  Likert  scale  was  used  to 


gather  information  from  the  respondents  concerning  their 
perception  of  the  usefulness  of  the  current  algorithm  that 
the  AFALC  uses  to  determine  the  status  of  their  acquisition 
programs. 

Question  number  25  through  question  number  29  were 
structured  open  ended  questions  requiring  the  respondents  to 
assign  weights  to  the  different  ILS  elements  in  the 
different  phases  of  the  acquisition  process.  They  were  used 
to  gather  information  concerning  the  relative  value  of  the 
different  ILS  elements  and  phases  to  the  acquisition 
process. 


Survey  Pretest 

The  survey  instrument  was  pre-tested  using  logistics 
management  specialists  assigned  to  APALC/LS  and  also 
logistics  management  specialists  who  had  just  completed  the 
Deputy  Program  for  Management  Logistics  (AFALC  001)  course 
conducted  by  the  AFALC.  The  purpose  of  the  pre-test  was  to 
determine  the  amount  of  time  required  to  complete  the 
survey,  to  evaluate  the  readability  of  the  instrument,  to 
improve  the  instrument,  aiid  to  check  the  content  validity  of 
the  survey  instrument.  Based  on  the  feedback  from  the 
pretest,  fifteen  question  concerning  the  ALHZS  data  base  and 
its  purpose  were  removed  from  the  survey.  In  the  place  of 
these  questions,  two  questions  were  asked  about  the  role 
that  the  ALHIS  data  base  should  play  in  the  assessment  of 
acquisition  programs. 


Analysis  Methodology 

Questions  1  through  6  of  the  survey  were  multiple 
choice  questions.  They  were  used  to  gather  and  analyze  the 
demographic  data  that  was  used  to  classify  the  respondents. 
For  these  questions,  a  frequency  count  for  each  category  was 
used.  The  results  of  this  analysis  are  in  chapter  IV. 

A  Chi  Square  test  of  independence  was  conducted  on 
question  number  7  through  24  to  determine  any  significant 
differences  between  the  categories  based  on  program 
orientation  for  acquisition  logisticians  and  support 
logisticians.  The  mode  and  the  median  were  also  computed 
for  each  program  orientation  for  both  acquisition 
logisticians  and  support  logisticians.  The  mode  is  the 
measurement  that  occurs  with  the  greatest  frequency  in  the 
data  set  (26:58).  The  median  of  a  data  set  is  a  number  such 
that  half  the  measurements  fall  below  the  median  and  half 
fall  above  (26:64).  The  mode  was  used  to  show  how  strongly 
the  respondents  feel  about  the  specific  issues  whereas  the 
median  was  used  to  provide  a  rank  order  of  the  respondents. 
The  results  of  this  analysis  are  in  chapter  IV. 

The  information  from  question  number  25  through 
question  number  28  was  analyzed  by  using  a  2x5  factorial 
design  in  order  to  determine  if  there  was  any  significant 
difference  at  the  .01  alpha  level  between  the  weights 
assigned  for  the  different  ILS  elements  by  acquisition 
logisticians  and  support  logisticians,  based  on  program 
orientation  and  the  phase  of  the  acquisition  process.  A 
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factorial  design  is  a  method  for  selecting  the  treatments  or 
the  factor  levels  to  be  included  in  an  experiment.  A 
complete  factorial  design  is  one  in  which  observations  are 
made  for  every  combination  of  factors.  In  order  to  conduct 
an  analysis  of  variance  it  is  necessary  to  assume  that  the 
population  variances  are  equal  (26:701-702).  In  this  study 
the  factors  that  were  of  value  were  the  end  item  management 
orientation  of  the  respondent  and  whether  the  respondent  was 
a  support  logistician  or  an  acquisition  logistician. 

Because  the  shape  of  the  population  probability  distribution 
was  unknown  a  nonparametric  test  was  used  to  analyze  any 
differences  between  the  rank  order  of  the  ILS  elements.  No 
assumptions  concerning  the  variance  of  the  sample  are 
required  for  the  nonparametric  counterpart  to  the  analysis 
of  variance,  the  Friedman  Fr  test  (28:139-146).  When  using 
the  Friedman  Fr  Test  for  a  randomized  block  design  certain 
assumptions  are  made.  They  are  as  follows: 

1.  The  treatments  are  randomly  assigned  to  an 
experimental  unit  within  the  blocks. 

2.  The  measurements  can  be  ranked  within  blocks. 

3.  Either  the  number  of  blocks  or  the  number  of 
treatments  should  exceed  five  for  the  approximation  to  be 
adequate  (23; 26: 763). 

Using  the  ILS  elements  and  the  five  end  item  management 
orientations,  these  assumptions  were  adhered  to  for  this 
study.  To  assist  in  solving  the  investigative  questions, 
respondents  with  at  least  24  months  of  logistics  experience 
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were  used  as  a  panel  of  experts.  The  24  month  level  was 
chosen  based  on  pretest  scores  for  the  Deputy  Program 
Manager  for  Logistics  course.  Pretest  scores  indicated  that 
logisticians  with  at  least  24  months  of  experience  generally 
scored  at  least  one  standard  deviation  above  the  mean. 
Although  there  have  been  no  formal  studies  to  explain  why 
logisticians  with  at  least  24  months  of  experience  score 
higher,  it  is  assumed  that  knowledge  of  the  ILS  elements  is 
the  reason  (18). 

A  computer  program  was  built  to  conduct  a  2x5  analysis 
of  variance  on  the  responses  to  question  nusd>er  25  through 
question  number  28  for  the  experts.  A  Friedman  Fr  test  for 
a  randomized  block  design  was  also  used.  The  results  of 
this  analysis  are  in  chapter  IV. 

The  information  from  question  number  29  was  analyzed  by 
using  a  2x5  factorial  design  in  order  to  determine  if  there 
was  any  significant  difference,  at  the  .01  alpha  level, 
between  the  weights  assigned  for  the  different  phases  of  the 
acquisition  process  by  acquisition  logisticians  and  support 
logisticians,  based  on  program  orientation.  A  Friedman  Fr 
test  for  a  randomized  block  design  was  also  used  to  analyze 
the  data  at  the  .01  alpha  level  in  order  to  determine  any 
differences  in  the  rank  order  of  the  phases.  The  results  of 
this  analysis  are  in  chapter  IV. 


17.  Findings  and  Analysis 


Introduction 

The  purpose  of  this  study  was  to  examine  the 
perceptions  that  logisticians  have  of  the  effectiveness  of 
the  current  algorithm  used  by  AFALC  and  to  determine  what 
weights  should  be  applied  to  the  individual  ILS  elements  in 
the  various  phases  of  the  system  acquisition  process  in 
order  to  improve  the  algorithm.  This  chapter  presents  a 
summary  of  the  survey  responses  and  an  analysis  of  the 
collected  data.  It  also  addresses  the  four  investigative 
questions  outlined  in  the  first  chapter  based  on  an  analysis 
of  the  information  provided  by  the  respondents. 

The  chapter  is  divided  into  five  main  parts:  (a)  sample 
response  summary;  (b)  presentation  of  the  demographic  data; 
(c)  a  perception  of  the  usefulness  of  ILS  elements  in  the 
current  algorithm;  (d)  the  relative  value  of  the  different 
ILS  elements  during  the  different  acquisition  phases  and  the 
relative  value  of  the  different  acquisition  phases  to  the 
acquisition  process;  and  (e)  a  summary  of  the  results  for 
the  investigative  questions. 

Sample  Response  Summa ry 

The  sample  consisted  of  71  Air  Force  officers  from  2nd 
Lieutant  (01)  to  Colonel  (06)  with  a  duty  AFSC  of  66XX  or 
64XX  and  122  civilians  from  GS-11  to  GH-IS  assigned  to 
logistics  activities  within  the  AFSC  product  divisions,  AFLC 
Air  Logistics  Centers,  or  AFALC.  Of  the  372  surveys  mailed. 
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11  were  returned  because  the  people  had  changed 
organizations.  Seven  of  the  surveys  were  returned  with 
notes  indicating  that  the  respondent  did  not  have  sufficient 
knowledge  to  complete  the  survey.  This  reduced  the  sample 
size  to  354.  Of  the  354  remaining  surveys,  234  were 
returned  by  the  indicated  return  date.  However,  16  of  the 
surveys  weris  not  returned  until  after  the  data  base  was 
built.  Of  the  remaining  surveys  25  were  not  usable  because 
of  being  incorrectly  filled  out.  This  represents  a  response 
rate  of  64  percent.  The  Air  Force  Manpower  and  Personnel 
Center  (APMPC)  reports  response  rates  around  60  percent  for 
Air  Force  surveys  (23).  Based  on  this,  the  response  rate 
for  this  survey  was  considered  satisfactory. 

The  survey  contained  demographic  questions,  ZLS 
perception  questions,  and  ILS  weight  questions.  A  copy  of 
the  survey  is  in  Appendix  A. 

Demographic  Data 

The  demographic  data  was  used  to  group  the  respondents 
as  either  being  an  acquisition  logistician  or  as  being  a 
support  logistician.  The  data  was  also  used  to  group  the 
respondents  into  one  of  the  following  five  end  item 
management  orientations: 

1.  Aircraft  Systems 

2.  Electronic  Systems 

3.  Missile  Systems 

4.  Munition  Systems 

5.  Space  Systems 
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Question  number  1  through  question  number  6  were  used 


to  gather  demographic  data  to  classify  the  respondents. 
These  questions  required  the  respondents  to  choose  a 
multiple  choice  response. 

The  first  section  of  the  survey  dealt  with 
demographic  questions.  The  results  for  the  number  of 
respondents  based  on  program  orientation  are  in  Table  1. 

TABLE  1 

RESPONDENTS  BASED  ON  PROGRAM  ORIENTATION 


ACQUISITION 

MILITARY 

CIVILIAN 

TOTAL 

AIRCRAFT  SYSTEMS 

23 

37 

60 

ELECTRONIC  SYSTEMS 

17 

15 

32 

MISSILE  SYSTEMS 

4 

9 

13 

MUNITION  SYSTEMS 

4 

8 

12 

SPACE  BASED  SYSTEMS 

_2 

10 

12 

20 

79 

129 

SUPPORT 

MILITARY 

CIVILIAN 

TOTAL 

AIRCRAFT  SYSTEMS 

5 

15 

20 

ELECTRONIC  SYSTEMS 

6 

10 

16 

MISSILE  SYSTEMS 

3 

5 

8 

MUNITION  SYSTEMS 

3 

9 

12 

SPACE  BASED  SYSTEMS 

_4 

_4 

_8 

21 

43 

64 
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Table  2  depicts  the  length  of  time  worked  in  logistics 


by  end  item  management  orientation  for  both  acquisition 
logisticians  and  support  logisticians. 


TABLE  2 

TOTAL  TIME  WORKED  IN  LOGISTICS 


ACQUISITION  0-12  MO 

AIRCRAFT  SYSTEM  4 

ELECTRONIC  SYSTEM  4 

MISSILE  SYSTEM  0 

MUNITION  SYSTEM  1 

SPACE  BASED  SYSTEMS  0 

9 


12-24  MO  24-36  MO  OVER  36  MO 


18  9  29 

14  2  12 

3.4  6 

2  4  5 

_3  _1  _8 

40  20  60 


SUPPORT  0-12  MO 

AIRCRAFT  SYSTEMS  3 

ELECTRONIC  SYSTEMS  3 

MISSILE  SYSTEMS  1 

MUNITION  SYSTEMS  3 

SPACE  BASED  SYSTEMS  _1 

11 


12-24  MO  24-36  MO  OVER  36  MO 


2  6  9 

0  7  6 

0  4  3 

3  15 

1  _2  _4 

6  20  27 


ILS  Perception  Data 

Question  number  7  through  question  number  24  required 
the  respondents  to  answer  using  a  seven  point  Likert  scale. 
Table  3  depicts  the  name  assigned  to  each  numeric  value  for 
the  Likert  scale. 
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TABLE  3 


LIKERT  SCALE  VALUES 
7  -  Strongly  Agree 
6  -  Moderately  Agree 
5  >  Agree 

4  -  Neither  Agree  Nor  Disagree 
3  Disagree 
2  >  Moderately  Disagree 
1  -  Strongly  Disagree 

For  these  questions  the  mode  and  median  are  of  the  most 
value  in  determining  the  perception  of  the  respondents.  The 
answers  to  these  questions  were  expressed  in  terms  of 
frequencies  of  response.  The  data  was  measured  in  terms  of 
the  number  of  individuals  who  were  in  each  of  the  different 
categories.  In  order  to  determine  if  the  frequencies  of  the 
different  categories  differ  from  what  could  be  attributed  to 
chance,  a  Chi  square  analysis  was  used.  The  Chi  Square  test 
was  used  to  determine  any  significant  difference  for  support 
logisticians  and  acquisition  logisticians.  The  Chi  Square 
test  was  also  used  to  determine  if  there  was  any  significant 
difference  between  the  respondents  based  on  end  item 
management  orientation. 

The  data  was  analyzed  by  using  the  basic  programming 
language  and  the  VP  Planner  spreadsheet  program.  The 
frequency  of  response  for  each  group,  for  each  question  on 
the  survey  instrument,  was  computed  by  conducting  an 
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analysis  on  each  question  using  the  VP  Planner  spreadsheet 
program.  The  median  and  mode  were  also  computed  for 
questions  involving  interval  and  ratio  data  (23).  Input  for 
the  investigative  questions  were  provided  based  on  the  ILS 
perception  questions.  A  5x7  Chi  Square  Contingency  table 
using  a  .05  significance  level,  was  used  to  determine  if 
there  is  any  significant  difference  between  or  within  the 
groups  concerning  the  perceptions  of  acquisition 
logisticians  and  support  logisticians  relative  to  the 
investigative  questions.  The  results  of  this  analysis  are 
in  appendix  B. 

A  brief  summary  of  the  responses  to  the  questions  in 
this  section  is  provided  below.  Following  the  written 
explanation  is  a  quantitative  summary  showing  the  mode  and 
median  for  each  question  along  with  the  results  of  the  Chi 
Square  for  that  question. 

Question  number  7  through  question  number  9  were 
designed  to  determine  the  extent  to  which  acquisition 
logisticians  and  support  logisticians  view  the  importance  of 
the  ILS  elements  in  assessing  acquisition  program  status. 

The  results  of  these  questions  were  nearly  identical  with 
each  other.  The  mode  and  median  reported  for  each  of  the 
questions  for  both  support  logisticians  and  acquisition 
logisticians  tend  to  indicate  that  logisticians  perceive  the 
status  of  the  integrated  logistics  support  elements  to  be  a 
major  player  in  determining  the  status  of  an  acquisition 
program.  The  results  for  question  number  7  through  question 


number  9  show  that  69.8%  of  the  acquisition  logisticians  and 
79.8%  of  the  support  logisticians  agree  that  the  ILS 
elements  are  an  important  part  of  assessing  program  status. 
Based  on  this  premise  the  ILS  element  weights  are  a  major 
factor  in  determining  the  status  of  an  acquisition  program. 

Q7.  The  integrated  logistics  support  elements  form  the 
major  planning  components  of  an  acquisition  program. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  17.43  36.42 

SUPPORT  5  5  21.99  36.42 

Q8.  Integrated  Logistics  Support  elements  are  used  to 
achieve  the  required  support  capability  at  an  affordable 
life  cycle  cost. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  17.42  36.42 

SUPPORT  5  5  23.67  36.42 

Q9.  Integrated  Logistics  Support  elements  accurately 
reflect  the  status  of  an  acquisition  program. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  17.43  36.42 

SUPPORT  5  5  21.99  36.42 
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Question  number  10  was  designed  to  determine  the 
perception  o£  logisticians  concerning  the  rank  order  of  the 
integrated  logistics  support  elements  based  on  the  phase  of 
the  acquisition  process. 

The  rank  order  of  the  IL5  elements  being  phase 
independent  was  agreed  to  by  67.2%  of  the  support 
logisticians  and  by  57.2%  of  the  acquisition  logisticians. 
Based  on  the  weights  assigned  by  the  logisticians  there  is  a 
difference  of  rank  ordering  the  ILS  elements  in  the 
different  phases  of  the  acquisition  process.  However,  based 
on  the  results  of  the  analysis  of  variance  for  question 
number  24  through  question  number  28  these  differences  are 
not  significant  at  the  .01  alpha  level.  Also,  based  on  the 
Friedman  Fr  non  parametric  test  there  was  no  significant 
difference  between  the  rank  order  of  the  ILS  elements  based 
on  the  phase  of  the  program  at  the  .01  alpha  level. 

QIO.  The  rank  order  of  the  integrated  logistics  support 
elements  are  phase  independent. 

MEDIAN  NODE  CRI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  15.03  36.42 

SUPPORT  5  5  23.65  36.42 

Question  number  11  was  designed  to  determine  the 
perception  of  logisticians  concerning  the  rank  order  of  the 
integrated  logistics  support  elements  based  on  the  program 
orientation. 


Both  Support  logisticians  and  acquisitions  logisticians 
tended  to  moderately  agree  that  the  rank  order  of  the  ILS 
elements  are  program  independent.  Of  the  support 
logisticians,  59.4%  agreed  with  this  statement  while  65.1% 
of  the  acquisition  logisticians  agreed.  The  median  and  mode 
reported  for  this  question  by  both  acquisition  logisticians 
and  support  logisticians  tend  to  indicate  that  logisticians 
perceive  the  rank  order  of  the  ILS  elements  to  be  program 
independent.  Based  on  the  weights  assigned  by  the 
logisticians  there  is  a  slight  difference  of  rank  ordering 
the  ILS  elements  for  different  program;^.  However,  based  on 
the  analysis  of  variance  for  question  nufld>er  24  through 
question  nu8d>er  28  these  differences  are  not  significant  at 
the  .01  alpha  level.  Also,  based  on  the  Friedman  non 
parametric  test  there  was  no  significant  difference  between 
the  rank  order  of  the  ILS  elements  based  on  the  phase  of  the 
program  at  the  .01  alpha  level. 

Qll.  The  rank  order  of  the  integrated  logistics  support 
elements  are  program  independent. 


MEDIAN 

MODE 

CBI  SQUARE 

.05  SIGNIFICANCE 

ACQUISITION 

5 

5 

17.72 

36.42 

SUPPORT 

5 

5 

22.21 

36.42 

Question  number  12  through  question  number  15  were 
designed  to  determine  if  logisticians  perceive  the 
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integrated  logistics  support  elements  to  be  equally  weighted 
in  the  different  acquisition  phases.  Because  the  answers  to 
these  questions  were  basically  the  same,  they  are  being 
reported  together.  Based  on  the  weights  assigned  by  the 
logisticians,  there  is  statistical  evidence  to  indicate  that 
the  weights  of  the  ILS  elements  for  any  of  the  phases  of  the 
acquisition  process  are  not  equal.  The  medians  and  modes 
reported  for  these  questions  by  both  acquisition 
logisticians  and  support  logisticians  also  tends  to  indicate 
that  logisticians  perceive  the  ILS  elements  to  not  be 
weighted  equally.  Of  the  support  logisticians  48.4% 
disagreed  and  43.8%  agree  with  this  statement.  This  appears 
to  be  a  result  of  the  munitions  and  space  based  logisticians 
disagreeing  while  the  other  logisticians  tended  to  agree. 

The  small  sample  size  for  these  two  groups  tends  to 
negatively  skew  the  data.  For  the  acquisition  logisticians, 
67.4%  agreed  that  the  integrated  logistics  support  elements 
should  be  equally  weighted  for  all  phases  of  the  acquisition 
process. 

Q12.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Concept  Exploration  phase. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  3  3  15.77  36.42 

SUPPORT  4  5  27.98  36.42 
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Q13.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Demonstration  Validation  phase. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  3  3  15.77  36.42 

SUPPORT  4  4  27.98  36.42 

Q14.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Full  Scale  Development  phase. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  3  3  15.77  36.42 

SUPPORT  3  3  27.98  36.42 

Q15.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Production  Deployment  phase. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  3  3  15.77  36.42 

SUPPORT  3  3  27.98  36.42 

Questions  16  through  18  were  designed  to  determine  to 
what  extend  logisticians  view  the  current  methods  of 
assessing  program  status  in  terms  of  effectiveness.  The 
median  and  mode  for  all  of  the  questions  for  both  support 
and  acquisition  logisticians  tend  to  indicate  that  there  is 
neither  agreement  nor  disagreement  with  the  methods  used  to 
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assess  the  status  of  acquisition  programs.  Of  the  support 
logisticians  40.4%  indicated  neither  agree  nor  disagree. 
Acquisition  logisticians  had  a  response  rate  of  40.3%  in 
this  category  with  35.7%  disagreeing.  There  is  a  slight 
leaning  toward  a  disagreement  that  the  current  methods  are 
effective  in  measuring  program  status. 

Q16.  The  current  method  used  by  APLC  to  measure  the  status 
of  an  acquisition  program  is  effective. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  4  4  16.76  36.42 

SUPPORT  4  4  18.79  36.42 

Q17.  The  current  method  used  by  AFSC  to  measure  the  status 
of  an  acquisition  program  is  effective. 

MEDIAN  MODE  CBI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  4  4  18.79  36.42 

SUPPORT  4  4  16.76  36.42 

Q18.  The  current  method  used  by  AFALC  to  measure  the  status 
of  an  acquisition  program  is  effective. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  4  4  18.79  36.42 

SUPPORT  4  4  16.76  36.42 
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Early  in  1988,  AFSC/PL  distributed  a  policy  letter  to 
the  AFSC  product  division's  program  offices  outlining 
guidance  that  should  be  used  in  assessing  an  individual  ILS 
element  color  code.  Questions  19  through  22  were  designed 
to  determine  how  logisticians  viewed  this  guidance. 

Question  number  19  tends  to  indicate  that  both  support 
logisticians  and  acquisition  logisticians  view  the  current 
ratings  as  being  too  subjective.  As  a  result,  the  responses 
to  question  number  20  through  22  indicate  a  strong  positive 
agreement  because  of  the  structure  that  is  provided. 

For  support  logisticians,  70.3%  agreed  that  the  there  is  too 
much  subjectivity  in  the  current  ratings  and  58.1%  of  the 
acquisition  logisticians  also  agreed. 

Q19.  The  current  Integrated  Logistics  Support  ratings  are 
too  subjective. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  13.87  36.42 

SUPPORT  5  5  24.14  36.42 


Q20.  An  ILS  element  with  minor  problems  that  has  solutions 
available  should  be  rated  green. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  16.26  36.42 

SUPPORT  5  5  18.86  36.42 
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Q21.  An  ILS  element  with  minor  to  severe  problems,  that  has 
potential  solutions,  should  be  rated  yellow. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  15.26  36.42 

SUPPORT  5  5  18.86  36.42 

Q22.  An  element  with  serious  or  potentially  serious 
problems,  that  require  higher  headquarters  assistance, 
should  be  rated  red. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  5  5  13.87  36.42 

SUPPORT  5  5  24.14  36.42 

Question  number  23  and  question  number  24  .were  designed 
to  determine  what  logisticians  perceived  the  role  of  the 
ALMIS  data  base  to  be,  as  it  pertained  to  a  management 
information  system  and,  as  it  pertained  to  a  decision 
support  system.  The  data  tends  to  indicate  that  most 
logisticians  are  neither  in  agreement  nor  disagreement  as  to 
the  role  of  the  ALNIS  data  base  being  used  as  only  a 
management  information  system  or  a  decision  support  tool. 

The  39.6%  of  acquisition  logisticians  who  agreed  with  these 
questions  is  offset  by  the  33.3%  who  disagreed.  Support 
logisticians  showed  similar  results  with  21.9%  agreeing  and 
32.8%  disagreeing. 
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Q23.  The  purpose  of  an  Logistics  Management  information 
System  is  to  function  as  a  central  information  repository 
for  Logistics  information. 

MEDIAN  MODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  4  4  20.72  36.42 

SUPPORT  4  4  14.79  36.42 

Q24.  T'.ie  purpose  of  a  Logistics  Management  Information 
System  is  to  function  as  a  Decision  Support  System. 

MEDIAN  NODE  CHI  SQUARE  .05  SIGNIFICANCE 
ACQUISITION  4  4  20.72  36.42 

SUPPORT  4  4  14.79  36.42 

ILS  Relative  Value  Data 

The  questions  in  the  third  part  of  the  survey  were 
designed  to  determine  the  views  of  logisticians  on  the 
importance  of  each  of  the  Integrated  Logistics  Support 
elements,  for  each  of  the  acquisition  phases,  they  were 
also  used  to  determine  the  relative  weights  for  each  phase 
of  the  acquisition  process,  for  both  support  logisticians 
and  acquisition  logisticians.  The  responses  of  logisticians 
that  were  classified  as  experts  were  used  to  determine  the 
mean  weights  used  for  the  analysis  of  the  data  in  this 
section.  An  expert  was  classified  as  a  person  who  has 
worked  a  minimum  of  two  years  in  the  logistics  field.  This 
figure  was  arrived  at  based  on  results  of  the  Deputy  Program 
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Manager  for  Logistics  (AFALC  001}  course  conducted  by  AFALC 
and  based  on  the  recommendation  of  the  Assistant  to  the 
Commander  for  AFALC.  For  question  number  25  through 
question  number  28,  the  following  instructions  were  given  to 
the  logisticians:  Based  on  their  relative  importance  to 
you,  rank  order  and  weight  the  Integrated  Logistics  Support 
elements,  for  the  various  phases,  by  distributing  100  points 
among  each  of  the  10  elements.  Do  not  equally  weight  them 
unless  you  believe  there  is  no  difference  between  them. 

Question  29  was  designed  to  determine  the  relative 
weights  for  each  phase  of  the  acquisition  process  for  both 
support  logisticians  and  acquisition  logisticians.  The 
following  instructions  were  given  to  the  logisticians: 

Based  on  their  relative  importance  to  you,  rank  order  and 
weight  the  acquisition  phases  by  distributing  100  points 
among  the  phases.  Do  not  equally  weight  them  unless  you 
believe  there  is  no  difference  between  them. 

For  question  number  25  through  question  number  28  the 
responses  were  input  to  a  spreadsheet  utilizing  VP  Planner. 

A  mean  weight  for  each  element  was  computed  for  each  group. 
The  mean  weights  for  the  individual  elements  were  derived  by 
using  the  "AT  AVG"  command.  The  standard  deviations  and  the 
variances  for  the  different  elements  were  computed  by  using 
the  "AT  STDS"  and  "AT  VAR"  commands.  In  order  to  determine 
if  there  was  a  difference  in  the  weights  assigned  for  the 
different  ILS  elements  by  support  logisticians  and 
acquisition  logisticians,  based  on  end  item  management 
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orientation,  a  2x5  analysis  of  variance  factorial  design,  at 
the  .01  alpha  level,  was  conducted  for  each  of  the  ILS 
elements,  in  each  phase,  for  both  support  logisticians  and 
acquisition  logisticians.  The  results  of  this  analysis  are 
in  appendix  C. 

In  the  factorial  design,  acquisition  logisticians  for 
the  five  program  management  orientations  were  designated  as 
Fa.  Support  logisticians  for  the  five  program  management 
orientations  were  designated  as  Fb.  The  designation  of  Faxb 
was  used  to  analyze  the  interaction  between  the  support 
logisticians  and  the  acquisition  logisticians.  For  each  of 
the  ILS  elements,  in  each  phase  of  the  acquisition  process, 
the  following  null  hypothesis  was  tested. 

Ho:  There  is  no  difference  between  the  weights  assigned 
by  acquisition  and  support  logisticians,  for  this  element, 
based  on  program  orientation. 

The  observed  F  did  not  exceed  the  F  .01  value  for  any 
of  the  elements.  Therefore,  we  can  not  reject  the  null 
hypothesis  of  no  difference  between  or  within  the  groups. 

The  results  of  the  F  tests  tend  to  indicate  there  is  no 
difference  between  the  individual  ILS  elements  at  the  .01 
alpha  level.  The  results  of  the  Friedman  Fr  test  tend  to 
indicate  there  is  no  difference  between  the  rank  ordering  of 
the  ILS  elements  in  the  different  phases.  This  result 
coincides  with  the  results  of  question  number  12  through 
question  number  15.  Appendix  D  summarizes  the  results  of 
the  analysis. 
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For  question  number  29  an  analysis  o£  variance  was 
conducted  for  each  phase  of  the  acquisition  process  for  both 
support  logisticians  and  acquisition  logisticians.  The 
analysis  of  the  data  tends  to  indicate  that  the  weight  for 
the  Concept  Exploration  phase  is  program  dependent.  The 
mean  weight  for  the  phase  for  acquisition  logisticians  was 
24.01  with  a  range  of  20.5  to  28.9  and  23.78  with  a  range  of 
10.56  to  35.93  for  support  logisticians.  The  analysis  of 
variance  for  acquisition  logisticians  shows  an  P  test  value 
of  4.33.  The  analysis  of  variance  for  support  logisticians 
shows  an  F  test  value  of  7.33.  These  values  exceed  the 
critical  value  of  F  at  the  .01  significance  level  for  Fa  and 
Fb  but  not  for  the  interaction  term  of  Faxb.  However,  the 
large  variance  associated  with  the  samples,  coupled  with  the 
small  sample  population,  tend  to  negatively  skew  the  data. 

For  the  other  phases  of  the  acquisition  process  the 
data  tends  to  indicate  that  there  is  no  difference  between 
the  mean  weights  based  on  program  orientation  for  either 
support  logisticians  or  acquisition  logisticians.  This  data 
is  summarized  in  appendix  E. 

Investigative  Questions  Summary 

1.  How  do  logisticians  perceive  the  phases  of  the 
acquisition  process  should  be  weighted? 

The  results  of  the  analysis  tend  to  indicate  that  the 
Demonstration/Validation  phase  and  the  Full  Scale 
Development  phase  are  the  most  important  followed  by  the 
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concept  exploration  phase.  A  complete  result  of  this 
analysis  is  in  appendix  E. 

2.  How  do  logisticians  perceive  the  ILS  elements 
should  be  rank-ordered? 

The  rank  order  of  the  ILS  elements  in  the  different 
phases  are  equal.  However,  the  individual  weights  are  not 
equal.  The  ILS  elements  of  design  interface,  maintenance 
planning,  support  equipment,  and  technical  data  are  the  top 
four  ranked  ILS  elements  in  the  Concept  Exploration  phase, 
Demonstration/Validation  phase,  and  the  Full  Scale 
Development  phase.  However,  in  the  Production/Deployment 
phase,  technical  data,  maintenance  planning,  supply  support, 
and  support  equipment  are  the  top  ranked  ILS  elements.  The 
results  of  this  analysis  are  in  appendix  F. 

3.  What  is  the  perceived  relationship  between  the 
weights  for  ILS  elements  in  one  phase  compared  to  the  other 
phases? 

Based  on  the  weights  assigned  by  both  acquisition  and 
support  logisticians  there  is  evidence  to  indicate  that  the 
weights  of  the  ILS  elements  for  any  of  the  phases  of  the 
acquisition  process  are  not  equal.  The  medians  and  modes 
reported  for  this  question  by  both  acquisition  logisticians 
and  support  logisticians  also  tends  to  indicate  that 
logisticians  perceive  the  ILS  elements  to  not  be  weighted 
equally. 

4.  Do  acquisition  logisticians  perceive  that  the 
weights  for  the  ILS  elements  are  program  independent? 
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To  answer  this  question  an  analysis  of  variance  was 
conducted  for  each  phase  of  the  acquisition  process.  The 
analysis  of  the  data  tends  to  indicate  that  the  weight  for 
the  Concept  Exploration  phase  is  program  dependent.  However, 
the  large  variance,  coupled  with  the  small  sample 
population,  tended  to  negatively  skew  the  data. 

For  the  other  phases  of  the  acquisition  process  the 
data  tends  to  indicate  that  there  is  no  difference  between 
the  mean  weights  for  the  ILS  elements  based  on  program 
orientation.  A  complete  result  of  the  analysis  is  in 
appendix  E. 
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V.  Summa ry  and  Recommendations 


Introduction 

This  chapter  presents  an  overview  and  brief  summary  of 
the  major  findings  of  the  study,  and  concludes  with 
recommendations  for  further  research. 

Overview 

Purpose  of  the  Study.  This  research  measured  the 
perceptions  that  logisticians  have  of  the  effectiveness  of 
the  current  method  of  evaluating  logistics  program  status 
and  to  determine  what  algorithm  should  be  used  to  measure 
the  logistics  status  of  acquisition  programs.  The  data 
collected  was  used  to  determine  if  acquisition  logisticians 
have  a  different  perception  of  the  effectiveness  of  the 
current  algorithm  than  support  logisticians  and  to  determine 
the  weights  to  be  used  for  the  ILS  elements  in  the  algorithm 
for  the  different  phases  of  the  acquisition  process.  The 
collected  data  was  analyzed  to  determine  any  relationships 
between  support  and  acquisition  logisticians  based  on  end 
item  management  orientation  and  the  phase  of  the  acquisition 
process. 

Survey  Summary 

Using  a  stratified  random  sample  based  on  current 
end  item  management  orientation,  data  was  collected  to 
determine  if  acquisition  logisticians  have  a  different 
perception  of  the  effectiveness  of  the  current  algorithm 
used  in  the  ALMZS  data  base  than  support  logisticians.  The 
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data  was  analyzed  to  determine  to  what  extent  the  current 
ALNIS  algorithm  is  perceived  to  be  effective  in  measuring 
logistics  program  status  for  acquisition  programs  and  to 
determine  what  weights  should  be  assigned  to  the  individual 
ILS  elements  in  the  various  phases  of  the  system  acquisition 
process  in  order  to  improve  the  algorithm. 

A  survey  was  mailed  to  252  acquisition  logisticians 
assigned  to  AFALC  and  matrixed  to  the  Air  Force  Systems 
Command's  (AFSC)  product  divisions  or  assigned  to  AFSC.  The 
survey  was  also  mailed  to  120  support  logisticians  assigned 
to  the  AFLC  Air  Logistics  Centers.  These  organizations  were 
chosen  because  their  item  management  orientation  is  similar 
to  the  item  management  orientation  of  the  AFSC  product 
divisions . 

Summary  of  Findings 

In  developing  a  new  algorithm  it  is  necessary  to 
compare  the  weights  assigned  by  the  logisticians  in  this 
study  to  the  existing  weights.  The  current  algorithm 
assigns  an  equal  weight  to  each  of  the  ILS  elements  across 
all  phases  of  the  acquisition  process.  The  weights  are 
tempered  somewhat  by  the  assignment  of  phase  weights.  The 
current  algorithm  also  uses  a  weight  for  the  Air  Force 
precedence  code.  However,  this  weight  is  more  of  a 
political  influence  than  a  major  factor  in  assessing 
logistics  considerations  for  program  status.  For  this 
reason,  this  study  did  not  consider  the  precedence  score. 
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The  factors  that  were  considered  were,  (1)  the 
weights  to  be  assigned  to  the  acquisition  phases;  (2)  the 
weights  to  be  assigned  to  the  individual  ILS  elements;  (3) 
the  rank  order  of  the  ILS  elements;  and  (4)  the  end  item 
program  orientation.  There  was  no  statistical  difference 
between  the  weights  assigned  for  the  different  phases  of  the 
acquisition  process.  This  is  not  to  imply  that  the  phases 
are  equal  but  rather  that  the  specific  program  phase  does 
not  influence  the  status  of  the  acquisition  process.  While 
there  was  no  statistically  significant  difference  for  the 
rank  order  of  the  ILS  elements  there  was  statistical 
evidence  to  indicate  that  the  ILS  elements  are  not  equally 
weighted.  The  observed  value  of  Fr  for  this  analysis  is 
3.36.  In  order  for  this  to  be  significant  at  the  .01  alpha 
level  would  require  a  Fr  of  11.3449.  The  observed  value  of 
Fr  does  not  exceed  the  Fr.Ol  value.  Therefore,  we  can  not 
reject  the  HO  of  no  difference  between  the  rank  order  of  the 
elements.  The  indication  from  this  analysis  is  that  there 
is  no  difference  between  the  rank  order  of  the  elements  in 
the  different  phases.  This,  coupled  with  the  results  of  the 
analysis  of  variance  in  appendix  D,  shows  that  because  there 
is  no  difference  in  the  weights  for  the  ILS  elements  based 
on  program  orientation  and  phase  of  the  acquisition  process, 
the  mean  weight  assigned  for  a  given  ILS  element  is  a  good 
indicator  of  the  weight  that  should  be  applied  in  the  given 
phase.  The  data  shows  that  the  status  of  an  acquisition 
program  can  be  assessed  by  using  the  following  algorithm. 
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1.  ZLS  Element  Score.  The  current  algorithm  utilizes  the 
following  color  coding  for  the  individual  ILS  elements: 

Green  -  0 
Yellow  «  1  to  5 
Red  ■■  6  to  9 

The  weight  of  20,  times  the  color  coding  weight  for  the 
specific  ILS  element,  equals  the  individual  ILS  element 
score.  The  sum  of  the  scores  equals  the  weight  assigned  for 
the  ILS  elements.  It  is  proposed  that  in  lieu  of  weighting 
the  ILS  elements  by  a  constant  weight,  that  they  be  weighted 
according  to  Table  4  based  on  the  phase  of  the  program. 


TABLE  4 

Weights  for  the  ILS  Elements 


ILS  ELEMENT 

CE 

DV 

PSD 

PRO 

Maintenance  Planning 

14.91 

14.74 

14.02 

13.02 

Manpower  and  personnel 

8.73 

8.09 

7.68 

8.03 

Supply  Support 

8.45 

9.08 

10.49 

12.19 

Support  Equipment 

10.39 

10.98 

11.17 

11.51 

Technical  Data 

9.03 

10.57 

11.94 

13.04 

Training  and  Training  Support 

7.19 

6.73 

7.46 

8.63 

Computer  Resources  Support 

7.94 

7.94 

7.84 

7.52 

Facilities 

8.02 

8.72 

8.35 

8.57 

Packaging,  Handling,  Storage, 

and  Transportation 

6.59 

6.65 

7.02 

7.94 

Design  Interface 

18.49 

16.62 

14.05 

9.52 
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2.  Air  Force  Precedence  score.  The  current  algorithm 
utilizes  the  following  precedence  score: 

Precedence  score  -  80  divided  by  the  Fad  weight  times  the 
Urgency  of  Need  designator  weight.  The  precedence  score 
serves  to  add  little  to  the  overall  point  value  upon  which 
the  rank-ordering  is  based.  It  is  proposed  to  eliminate 
this  facet  of  the  evaluation  from  the  algorithm. 

3.  Program  Phase  score.  The  current  algorithm  utilizes  the 
following  weights  for  the  different  acquisition  phases. 


Production  Deployment  Phase  50 
Full  Scal^  Development  Phase  25 
Demonstration/Validation  Phase  25 
Concept  Exploration  Phase  10 


However,  the  analysis  of  the  data  tends  to  indicate  that 
there  is  no  difference  in  the  importance  of  the  specific 
phase  of  the  acquisition  process.  As  a  result  it  is 
proposed  that  this  item  be  removed  also. 

Recommendations  for  Future  Research 

In  conducting  the  research  for  this  thesis  and  while 
analyzing  the  data,  it  became  clear  that  there  are  several 
factors  which  future  research  should  consider. 

Life  Cycle  Cost.  The  importance  of  total  cost  analysis 
has  been  one  of  the  key  concepts  identified  for  logistics 
managers.  Leading  authorities  in  the  field  of  logistics 
suggest  that  in  order  to  manage  the  logistics  function  of  an 
organization  in  an  efficient  manner,  it  is  necessary  to 
utilize  total  cost  analysis  (13:568-571). 


The  ability  to  accurately  forecast  and  identify  life 
cycle  costs  impacts  all  logistics  decisions.  These  existing 
cost  trade-offs  should  be  identified  as  soon  as  possible  in 
the  weapon  system  acquisition  process  to  enhance  the  Air 
Force's  ability  to  make  accurate  cost  benefit  comparisons. 
DODD  5000.1  requires  that  the  costs  associated  with 
acquiring  the  system,  ownership  costs,  and  system  readiness 
be  balanced  in  terms  of  the  mission  (5:1).  This  directive 
also  requires  that  supportability  be  considered  while  the 
acquisition  strategy  is  being  developed  and  during  each  of 
the  phases  of  the  acquisition  process  (2:1,7:1). 

Life  cycle  cost  management  is  the  control  of  future  cost 
impacts,  performance  requirements  and  schedule  constraints 
in  arriving  at  today's  logistic  decisions  for  support  of  the 
new  system.  As  such,  the  issue  of  life  cycle  cost  should  be 
considered  when  assessing  the  logistics  status  of  an 
acquisition  program. 

Regression  Analysis.  A  regression  analysis  should  be 
run  on  the  current  algorithm  as  well  as  the  proposed 
algorithm  to  determine  any  factors  or  ILS  elements  which  can 
be  deleted.  Using  the  acquisition  program  documentation 
maintained  by  AFALC/LS  and  historical  data  that  is  available 
in  the  ALHIS  data  base,  the  new  algorithm  could  be  analyzed 
to  determine  if  it  more  effectively  measures  the  status  of 
acquisition  programs. 

ILS  Ratings  Subjectivity.  The  issue  of  the 
subjectivity  of  the  color  coding  for  the  individual  ILS 


elements  appears  to  be  a  recurring  theme.  An  effort  to 
remove  this  subjectivity  was  attempted  by  AFALC/LS  with 
their  "Questions  and  Answers"  program.  In  order  to  develop 
an  effective  algorithm  it  is  necessary  to  first  develop  a 
method  of  controlling  the  evaluator  subjectivity. 
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Appendix  A 
Research  Survey 


FROM:  LS 

SUBJECT:  Research  Survey  (USAF  Survey  Control  Number  88-64 
Expires  30  September  1988) 

TO:  Survey  Participant 


1.  From  a  management  perspective,  the  Integrated  Logistics 
Support  elements  are  used  to  help  determine  the  status  of  an 
acquisition  program.  The  purpose  of  this  survey  is  to 
analyze  the  perception  of  the  logistics  community  relative  to 
this  perspective. 

2.  As  a  professional  in  the  field  of  logistics,  your 
response  to  this  survey  is  important  to  help  us  determine  the 
relationship  between  the  Integrated  Logistics  Support 
elements  and  the  status  of  an  acquisition  program. 

3.  Your  participation  in  this  research  is  voluntary  but 
your  assistance  would  be  greatly  appreciated.  If  you  would 
like  a  copy  of  the  results,  please  send  your  request  with 
your  completed  survey.  Direct  any  questions  you  have 
concerning  this  research  to  Jim  Hiller,  AFIT/LSG. 

4.  Please  complete  the  attached  survey  and  return  it  in  the 

enclosed  envelope  by  _ • 


2  Atch 

1.  Survey 

2.  Return  Envelope 


57 


A  Perception  of  the  Effectiveness  of  the  Acquisition 
Logistics  Algorithm  in  Measuring  Program  Status. 


DIRECTIONS: 

This  survey  contains  three  sections.  Section  I 
contains  demographic  questions.  Section  II  contains 
questions  concerning  your  view  on  the  use  of  the  Integrated 
Logistics  Support  elements  for  determining  program 
status.  Section  III- contains  questions  concerning  your 
views  on  the  importance  of  the  individual  ILS  elements. 

Please  answer  all  of  the  questions  for  Section  I  and 
Section  II  of  this  survey  by  marking  the  response  that  best 
represents  your  view.  If  you  do  not  find  a  response  that 
represents  your  view  use  the  one  that  is  closes  to  your 
view.  Nark  your  response  on  the  attached  machine-scored 
answer  sheet  using  a  number  2  pencil.  Answer  the  questions 
for  Section  III  directly  on  this  survey  form.  This  survey 
will  take  about  20  minutes  to  complete. 
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Section  I 


The  questions  in  this  section  are  to  obtain  information 
about  you  and  your  organization. 

1.  I  am  a  (an) 

1.  Officer 

2.  Enlisted 

3.  Civilian  (GS) 

4.  Civilian  (GH) 

2.  My  grade  level  is: 

1.  1-2 

2.  3-4 

3.5-6 

4.  7-8 

5.  9-10 

6.  11-13 

7.  14-15 

8.  SES 

3.  My  duty  AFSC/Series  is: 

1.  66XX 

2.  64XX 

3.  346 

4.  345 

5.  20XX 

6.  Other 

4.  The  total  time  I  have  worked  in  logistics  is: 

1.  Less  than  12  months 

2.  12  months  to  24  months 

3.  24  to  36  months 

4.  more  than  36  months 

5.  The  systems  I  primarily  work  with  are: 

1.  Aircraft  Systems 

2.  Electronic  Systems 

3.  Missile  Systems 

4.  Munition  Systems 

5.  Space  Based  Systems 

6.  Other 

6.  My  current  position  is: 

1.  DPML/ILSM 

2.  Acquisition  Logistics  Staff  Position 

3.  Support  Logistics  Staff  Position 

4.  Other 
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Section  II 


This  portion  of  the  questionnaire  is  to  determine  the 
degree  on  which  you  view  the  effectiveness  of  the  Integrated 
Logistics  Support  elements  in  measuring  program  status. 

Read  each  item  and  then  choose  the  statement  which  best 
describes  your  view  based  on  the  following  scale. 

7.  Strongly  Agree 
6.  Moderately  Agree 
5.  Agree 

4.  Neither  Agree  Nor  Disagree 
3.  Disagree 
2.  Moderately  Disagree 
1.  Strongly  Disagree 

7.  The  integrated  logistics  support  elements  form  the 
major  planning  components  of  an  acquisition  program. 

8.  Integrated  Logistics  Support  elements  are  used  to 
achieve  the  required  support  capability  at  an  affordable 
life  cycle  cost. 

9.  Integrated  Logistics  Support  elements  accurately 
reflect  the  status  of  an  acquisition  program. 

10.  The  rank  order  of  the  integrated  logistics  support 
elements  are  phase  independent. 

11.  The  rank  order  of  the  integrated  logistics  support 
elements  are  program  independent. 

12.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Concept  Exploration  phase. 

13.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Demonstration  Validation  phase. 

14.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Full  Scale  Development  phase. 

15.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Production  Deployment  phase. 

16.  The  current  method  used  by  AFLC  to  measure  the  status 
of  an  acquisition  program  is  effective. 

17.  The  current  method  used  by  AFSC  to  measure  the  status 
of  an  acquisition  program  is  effective. 

18.  The  current  method  used  by  AFALC  to  measure  the  status 
of  an  acquisition  program  is  effective. 
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Read  each  item  and  then  choose  the  statement  which  best 
describes  your  view  based  on  the  following  scale. 

7.  Strongly  Agree 
6.  Moderately  Agree 
5.  Agree 

4.  Neither  Agree  Nor  Disagree 
3.  Disagree 
2.  Moderately  Disagree 
1.  Strongly  Disagree 

19.  The  current  Integrated  Logistics  Support  ratings  are 
too  subjective. 

20.  An  ILS  element  with  minor  problems  that  has  solutions 
available  should  be  rated  green. 

21.  An  ILS  element  with  minor  to  severe  problems,  that  has 
potential  solutions,  should  be  rated  yellow. 

22.  An  element  with  serious  or  potentially  serious 
problems,  that  require  higher  headquarters  assistance, 
should  be  rated  red. 

23.  The  purpose  of  an  Logistics  Management  Information 
System  is  to  function  as  a  central  information  repository 
for  Logistics  information 

24.  The  purpose  of  a  Logistics  Management  Information 
System  is  to  function  as  a  Decision  Support  System. 
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Section  III 


The  questions  in  this  section  are  to  determine  your  views  on 
the  importance  of  each  of  the  Integrated  Logistics  Support 
elements  for  each  of  the  acquisition  phases.  Hark  your 
response  on  this  form. 


25.  Based  on  their  relative  importance  to  you,  rank  order 
and  weight  the  Integrated  Logistics  Support  elements,  for 
the  Concept  Exploration  phase,  by  distributing  100  points 
among  each  of  the  10  elements.  Do  not  equally  weight  them 
unless  you  believe  there  is  no  difference  between  them. 


1.  Maintenance  Planning 

2.  Manpower  and  Personnel 

3.  Supply  Support 

4.  Support  Equipment 

5.  Technical  Data 

6.  Training  and  Training  Support 

7.  Computer  Resources  Support 

8.  Facilities 

9.  Packaging,  Handling,  Storage, 
and  Transportation 

10.  Design  Interface 

TOTAL 


TM 


26.  Based  on  their  relative  importance  to  you,  rank  order 
and  weight  the  Integrated  Logistics  Support  elements,  for 
the  Demonstration/Validation  phase,  by  distributing  100 
points  among  each  of  the  10  elements.  Do  not  equally  weight 
then  unless  you  believe  there  is  no  difference  between  them. 


1.  Maintenance  Planning  _ 

2.  Manpower  and  Personnel  _ 

3.  Supply  Support  _ 

4.  Support  Equipment  _ 

5.  Technical  Data  _ 

6.  Training  and  Training  Support  _ 

7.  Computer  Resources  Support  _ 

8.  Facilities  _ 

9.  Packaging,  Handling,  Storage, 

and  Transportation  _ _ 

10.  Design  Interface 

TOTAL  TM 
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27.  Based  on  their  relative  importance  to  you,  rank  order 
and  weight  the  Integrated  Logistics  Support  elements,  for 
the  Full  Scale  Development  phase,  by  distributing  100  points 
among  each  of  the  10  elements.  Do  not  equally  weight  them 
unless  you  believe  there  is  no  difference  between  them. 


1.  Maintenance  Planning  _ 

2.  Manpower  and  Personnel  _ 

3.  Supply  Support  _ 

4.  Support  Equipment  _ 

5.  Technical  Data  _ 

6.  Training  and  Training  Support  ■  _ 

7.  Computer  Resources  Support  _ 

8.  Facilities  _ 

9.  Packaging,  Handling,  Storage, 

and  Transportation  _ 

10.  Design  Interface  _ _ 

TOTAL  nnr 


28.  Based  on  their  relative  importance  to  you,  rank  order 
and  weight  the  Integrated  Logistics  Support  elements,  for 
the  Production/Deployment  phase,  by  distributing  100  points 
among  each  of  the  10  elements.  Do  not  equally  weight  them 
unless  you  believe  there  is  no  difference  between  them. 

1.  Maintenance  Planning  _ 

2.  Manpower  and  Personnel  ^ 

3.  Supply  Support  _ 

4.  Support  Equipment  ^ 

5.  Technical  Data  _ 

6.  Training  and  Training  Support  _ 

7.  Computer  Resources  Support  _ 

8.  Facilities  ^ 

9.  Packaging,  Handling,  Storage, 

and  Transportation  _ 

10.  Design  Interface  _  _ 

TOTAL  nnj  T 


29.  Based  on  their  relative  importance  to  you,  rank  order 
and  weight  the  acquisition  phases  by  distributing  100  points 
among  the  phases.  Do  not  equally  weight  then  unless  you 
believe  there  is  no  difference  between  then. 

1.  Concept  Exploration  _ ^ 

2.  Demonstration/Validation  _ 

3.  Full  Scale  Development  _ 

4.  Production/Deployment  _ 


TOTAL  100 


Thank  you  for  your  time  and  assistance. 
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Appendix  B 
ILS  Perception  Data 

Q7.  The  integrated  logistics  support  elements  form  the 
major  planning  components  of  an  acquisition  program. 
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Q8.  Integrated  Logistics  Support  elements  are  used  to 
achieve  the  required  support  capability  at  an  affordable 
life  cycle  cost. 
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Q9.  Integrated  Logistics  Support  elements  accurately 
reflect  the  status  of  an  acquisition  program. 
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QlO.  The  rank  order  of  the  integrated  logistics  support 
elements  are  phase  independent. 


Frequency  of  Response  of  Acquisition  Logisticians 
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Qll.  The  rank  order  of  the  integrated  logistics  support 
elements  are  program  independent. 


Frequency  of  Response  of  Acquisition  Logisticians 
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Q12.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Concept  Exploration  phase. 

Frequency  of  Response  of  Acquisition  Logisticians 
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Q13.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Demonstration  Validation  phase. 


Frequency  of  Response  of  Acquisition  Logisticians 
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Q14.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Full  Scale  Development  phase. 

Frequency  of  Response  of  Acquisition  Logisticians 
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Q15.  The  Integrated  Logistics  Support  elements  should  be 
equally  weighted  in  the  Production  Deployment  phase. 
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Q16.  The  current  method  used  by  AFLC  to  measure  the  status 
of  an  acquisition  program  is  effective. 
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Q17.  The  current  method  used  by  AFSC  to  measure  the  status 
of  an  acquisition  program  is  effective. 

Frequency  of  Response  of  Acquisition  Logisticians 
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of  Response 

of  All 

.  Logisticians 

Data  Values 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

13 

13 

41 

78 

36 

12 

3 

4 

4 

Valid  Cases 

193 

Missing 

Cases 

0 
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Q18.  The  current  method  used  by  AFALC  to  measure  the  status 
o£  an  acquisition  program  is  effective. 

Frequency  of  Response  of  Acquisition  Logisticians 
Program  Orientation  1234567  Median  Mode 


Aircraft  Systems  66  13  22  9404  4 
Electronic  Systems  416  15  5014  4 
Missile  Systems  00254114  4 
Munition  Systems  03432003  3 
Space  Based  Systems  00173104  4 
Total  129 

Valid  Cases  129  Hissing  Cases  0 


Frequency  of  Response  of  Support  Logisticians 
Program  Orientation  1234567  Median  Mode 

Aircraft  Systems  104  10  4104  4 

Electronic  Systems  11354204  4 

Missile  Systems  01213014  5 

Munition  Systems  00451104  4 

Space  Based  Systems  11231104  4 

Total  64 

Valid  Cases  64  Hissing  Cases  0 

Frequency  of  Response  of  All  Logisticians 
Data  Values  1234567  Median  Mode 

13  13  41  78  36  12  3  4  4 

Valid  Cases  193  Hissing  Cases  0 
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Q19.  The  current  Integrated  Logistics  Support  ratings  are 
too  subjective. 


Frequency  of  Response  of  Acquisition  Logisticians 


Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  System*- 

0 

4 

13 

14 

15 

9 

5 

5 

5 

Electronic  Systems 

1 

1 

6 

5 

12 

4 

3 

5 

5 

Missile  Systems 

1 

0 

1 

3 

6 

0 

2 

5 

5 

Munition  Systems 

0 

0 

2 

1 

2 

5 

2 

6 

6 

Space  Based  Systems 

0 

0 

0 

2 

5 

3 

2 

5 

5 

Total 

129 

Valid  Cases 

129 

Missing 

Cases 

0 

Frequency  of 

Response 

of 

Support 

Logisticians 

Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

0 

0 

2 

4 

7 

2 

5 

5 

5 

Electronic  Systems 

0 

1 

1 

3 

6 

4 

1 

5 

5 

Missile  Systems 

0 

0 

0 

3 

1 

2 

2 

5 

4 

Munition  Systems 

0 

0 

1 

3 

3 

4 

0 

5 

6 

Space  Based  Systems 

0 

0 

0 

1 

4 

2 

2 

5 

5 

Total 

64 

Valid  Cases 

64 

Missing 

Cases 

0 

Frequency  i 

of  Response 

of  All  Logisticians 

Data  Values 

1 _ 

2 

3 

4 

5 

6 

7 

Median 

Mode 

2 

6 

26 

39 

61 

35 

24 

5 

5 

Valid  Cases 

193 

Missing 

Cases 

0 
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Q20.  An  ILS  element  with  minor  problems  that  has  solutions 
available  should  be  rated  green. 

Frequency  of  Response  of  Acquisition  Logisticians 
Program  Orientation  1  2  34567  Median  Mode 


Aircraft  Systems  2475  29  11  25  5 
Electronic  Systems  3135  12  6  2  5  5 
Missile  Systems  00137205  5 
Munition  Systems  11232214  4 
Space  Based  Systems  00324215  5 
Total  129 

Valid  Cases  129  Missing  Cases  0 


Frequency  of  Response  of  Support  Logisticians 
Program  Orientation  1234567  Median  Mode 


Aircraft  Systems  1023  10  315  5 

Electronic  Systems  1  0  2  18  40  5  5 

Missile  Systems  00123115  5 

Munition  Systems  00142315  4 

Space  Based  Systems  00124205  5 

Total  64 

Valid  Cases  64  Missing  Cases  0 

Frequency  of  Response  of  All  Logisticians 


Data  Values  1234567  Median  Mode 

8  6  23  30  81  36  9  5  5 

Valid  Cases  193  Missing  Cases  0 
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Q21.  An  ILS  element  with  minor  to  severe  problems,  that  has 
potential  solutions,  should  be  rated  yellow. 


Frequency  of  Response  of  Acquisition  Logisticians 


Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

2 

4 

7 

5 

29 

11 

2 

5 

5 

Electronic  Systems 

3 

1 

3 

5 

12 

6 

2 

5 

5 

Missile  Systems 

0 

0 

1 

3 

7 

2 

0 

5 

5 

Munition  Systems 

1 

1 

2 

3 

2 

2 

1 

4 

4 

Space  Based  Systems 

0 

0 

3 

2 

4 

2 

1 

5 

5 

Total 

129 

Valid  Cases 

129 

Missing 

Cases 

0 

Frequency  of 

Response 

of 

Support 

Logisticians 

Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

1 

0 

2 

3 

10 

3 

1 

5 

5 

Electronic  Systems 

1 

Q 

2 

1 

8 

4 

0 

5 

5 

Missile  Systems 

0 

0 

1 

2 

3 

1 

1 

5 

5 

Munition  Systems 

0 

0 

1 

4 

2 

3 

1 

5 

4 

Space  Based  Systems 

0 

0 

1 

2 

4 

2 

0 

5 

5 

Total 

64 

Valid  Cases 

64 

Missing 

Cases 

0 

Frequency  i 

of  Response 

of  All  Logisticians 

Data  Values 

1 _ 

2 

3 

4 

5 

6 

7 

Median 

Mode 

8 

6 

23 

30 

81 

36 

9 

5 

5 

Valid  Cases 

193 

Missing 

Cases 

0 
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Q22.  An  element  with  serious  or  potentially  serious 
problems,  that  require  higher  headquarters  assistance, 
should  be  rated  red. 

Frequency  of  Response  of  Acquisition  Logisticians 
Program  Orientation  1234567  Median  Mode 


Aircraft  Systems  2475  29  11  25  5 
Electronic  Systems  3135  12  625  5 
Nissile  Systems  00137205  5 
Munition  Systems  112  3'  2214  4 
Space  Based  Systems  00324215  5 
Total  129 

Valid  Cases  129  Hissing  Cases  0 


Frequency  of  Response  of  Support  Logisticians 
Program  Orientation  1234567  Median  Mode 


Aircraft  Systems  1023  10  315  5 

Electronic  Systems  10218405  5 

Missile  Systems  00123115  5 

Munition  Systems  00142315  4 

Space  Based  Systems  00124205  5 

Total  64 

Valid  Cases  64  Missing  Cases  0 

Frequency  of  Response  of  All  Logisticians 


Data  Values  1234567  Median  Mode 

8  6  23  30  81  36  9  5  5 

Valid  Cases  193  Missing  Cases  0 
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Q23.  The  purpose  of  an  Logistics  Management  Information 
System  is  to  function  as  a  central  information  repository 
for  Logistics  information 


Frequency  of  Response  of  Acquisition  Logisticians 


Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

7 

5 

10 

12 

18 

3 

5 

4 

5 

Electronic  Systems 

3 

1 

5 

10 

8 

4 

1 

4 

4 

Missile  Systems 

0 

1 

3 

5 

2 

0 

2 

4 

4 

Munition  Systems 

2 

1 

3 

4 

1 

0 

1 

4 

4 

Space  Based  Systems 

1 

2 

0 

4 

2 

2 

1 

4 

4 

Total 

129 

Valid  Cases 

129 

Missing 

Cases 

0 

Frequency  of 

Response 

of 

Support 

Logisticians 

Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

2 

0 

0 

12 

4 

0 

2 

4 

4 

Electronic  Systems 

0 

1 

3 

4 

6 

1 

1 

5 

5 

Missile  Systems 

1 

1 

1 

3 

0 

0 

2 

4 

4 

Munition  Systems 

0 

1 

2 

5 

3 

0 

0 

4 

4 

Space  Based  Systems 

0 

0 

2 

5 

2 

0 

0 

4 

4 

Total  64 

Valid  Cases  64  Hissing  Cases  0 

* 

Frequency  of  Response  of  All  Logisticians 
Data  Values  1234567  Median  Mode 

16  13  29  64  46  10  15  4  4 

Valid  Cases  193  Hissing  Cases  0 


Q24.  The  purpose  of  a  Logistics  Management  Information 
System  is  to  function  as  a  Decision  Support  System. 


Frequency  of  Response  of  Acquisition  Logisticians 


Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

7 

5 

10 

12 

18 

3 

5- 

4 

5 

Electronic  Systems 

3 

1 

5 

10 

8 

4 

1 

4 

4 

Missile  Systems 

0 

1 

3 

5 

2 

0 

2 

4 

4 

Munition  Systems 

2 

1 

3 

4 

1 

0 

1 

4 

4 

Space  Based  Systems 

1 

2 

0 

4 

2 

2 

1 

4 

4 

Total 

129 

Valid  Cases 

129 

Missing 

Cases 

0 

Frequency  of 

Response 

of 

Support 

Logi 

sticians 

Program  Orientation 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

Aircraft  Systems 

2 

0 

0 

12 

4 

0 

2 

4 

4 

Electronic  Systems 

0 

1 

3 

4 

6 

1 

1 

5 

5 

Missile  Systems 

1 

1 

1 

3 

0 

0 

2 

4 

4 

Munition  Systems 

0 

1 

2 

5. 

3 

0 

0 

4 

4 

Space  Based  Systems 

0 

0 

2 

5 

2 

0 

0 

4 

4 

Total 

64 

Valid  Cases 

64 

Missing 

Cases 

0 

Frequency  of  Response 

of  All  Logisticians 

Data  Veilues 

1 

2 

3 

4 

5 

6 

7 

Median 

Mode 

16 

13 

29 

64 

46 

10 

15 

4 

4 

Valid  Cases  193  Missing  Cases  0  I 
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Appendix  C 

ILS  Weights  by  Phase  and  Program  Orientation 

In  this  appendix  the  following  abbreviations  are  used 

NP  Maintenance  Planning 

HPP  Manpower  and  Personnel 

SS  Supply  Support 

SE  Support  Equipment 

TD  Technical  Data 

TTS  Training  and  Training  Support 

CRS  Computer  Resources  Support 

FAC  Facilities 

PHST  Packaging,  Handling,  Storage,  and 

Transportation 
DI  Design  Interface 

CE  Concept  Exploration  Phase 

DV  Demonstration/Validation  Phase 

FSD  Full  Scale  Development  Phase 

PRO  Production/Deployment  Phase 
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ILS  Weights  by  Phase  for  Aircraft  Logisticians 


ILS  ACQUISITION  SUPPORT 


ELEMENT 

CE 

DV 

PSD 

PRO 

CE 

DV 

FSO 

PRO 

HP 

18.66 

15.97 

12.11 

8.84 

15.33 

15.47 

14.47 

14.60 

MPP 

9.82 

9.21 

8.74 

9.53 

8.53 

8.67 

8.67 

9.47 

SS 

7.71 

8.63 

10.32 

12.82 

7.80 

8.47 

8.67 

10.20 

SE 

10.42 

10.63 

12.45 

12.21 

8.87 

10.40 

9.87 

12.53 

TD 

7.32 

9.00 

10.74 

13.24 

8.67 

10.00 

10.67 

10.67 

TTS 

5.61 

6.08 

8.13 

8.39 

9.00 

7.00 

7.13 

7.47 

CRS 

7.98 

8.50 

8.92 

9.18 

7.87 

9.35 

8.80 

7.53 

FAC 

6.34 

7.61 

7.95 

7.84 

7.27 

8.60 

8.20 

9.47 

PHST 

4.26 

5.45 

6.82 

7.74 

6.46 

6.13 

5.73 

7.40 

DI 

22.37 

18.03 

14.11 

9.95 

20.47 

17.00 

17.80 

10.67 

ILS  Weights  by  Phase  for  Electronic  Systems  Logisticians 


ILS  ACQUISITION  SUPPORT 


ELEMENT 

CE 

DV 

FSD 

PRO 

CE 

DV 

FSD 

PRO 

HP 

15.29 

14.43 

13.43 

14.64 

11.15 

12.31 

15.00 

16.92 

MPP 

7.14 

6.21 

7.86 

7.07 

8.46 

7.31 

6.15 

5.39 

SS 

7.21 

7.00 

11.86 

13.57 

10.00 

11.15 

11.54 

12.31 

SE 

10.86 

10.64 

10.93 

10.36 

11.92 

13.08 

12.69 

13.46 

TD 

9.93 

10.93 

12.28 

10.43 

9.62 

11.15 

11.92 

12.69 

TTS 

.5.64 

6.86 

8.36 

8.93 

8.46 

6.92 

6.15 

5.39 

CRS 

8.07 

8.50 

9.43 

7.36 

8.46 

7.69 

7.31 

e 

6.54 

FAC 

10.07 

8.36 

7.79 

8.43 

8.46 

7.69 

7.69 

8.46 

PHST 

5.57 

6.00 

6.14 

8.07 

8.46 

7.69 

6.92 

6.15 

DI 

20.21 

20.36 

11.93 

11.14 

14.23 

15.00 

14.63 

12.69 
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ILS  Weights  by  Phase  for  Missile  Systems  Logisticians 


ILS  ACQUISITION  SUPPORT 


ELEMENT 

CE 

DV 

PSD 

PRO 

CE 

DV 

FSD 

PRO 

MP 

13.00 

13.50 

13.90 

11.00 

11.43 

12.86 

12.14 

9.86 

MPP 

11.20 

9.00 

7.40 

7.00 

8.57 

7.86 

7.86 

8.71 

SS 

7.50 

8.00 

11.90 

14.00 

7.86 

9.29 

8.57 

10.71 

SE 

8.50 

8.50 

8.90 

10.00 

10.00 

12.14 

10.71 

12.14 

TD 

8.50 

11.00 

11.90 

14.00 

10.71 

11.43 

10.71 

13.57 

TTS 

9.30 

7.00 

7.90 

8.00 

7.86 

7.14 

8.57 

7.86 

CRS 

7.50 

5.00 

5.40 

6.50 

8.57 

8.57 

9.29 

6.43 

FAC 

9.50 

11.50 

9.90 

10.00 

10.71 

9.29 

9.29 

9.29 

PHST 

7.50 

8.00 

8.40 

9.50 

8.57 

7.43 

8.58 

8.57 

DI 

17.50 

18.50 

14.40 

10^00 

15.71 

14.00 

14.29 

12.86 

ILS  Weights  by  Phase  for  Munitions  Systems  Logisticians 


ILS  ACQUISITION  SUPPORT 


ELEMENT 

CE 

DV 

FSD 

PRO 

CE 

DV 

FSD 

PRO 

MP 

11.11 

10.56 

12.22 

9.11 

18.33 

21.67 

21.67 

25.83 

MPP 

8.33 

7.78 

7.78 

8.56 

8.33 

5.83 

5.00 

5.00 

SS 

10.00 

10.56 

11.11 

11.67 

10.83 

11.67 

11.67 

10.83 

SE 

12.78 

12.78 

11.67 

12.22 

8.33 

9.17 

8.33 

6.67 

TD 

8.89 

10.56 

13.33 

12.78 

12.50 

13.33 

15.00 

16.67 

TTS 

6.11 

7.22 

8.33 

10.56 

8.33 

6.67 

5.00 

5.83 

CRS 

8.33 

7.78 

6.67 

7.67 

7.50 

5.83 

5.00 

5.00 

FAC 

6.67 

12.78 

8.89 

10.22 

6.67 

7.50 

7.50 

8.33 

PHST 

5.56 

6.11 

7.00 

9.11 

7.50 

5.83 

5.83 

5.83 

DI 

22.22 

18.33 

13.00 

8.11 

11.67 

12.50 

15.00 

10.00 
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ILS  Weights  by  Phase  for  Spaced  Based  Systems  Logisticians 


f 


ILS  ACQUISITION  SUPPORT 


ELEMENT 

'  CE 

DV 

FSD 

PRO 

CE 

DV 

FSD 

PRO 

MP 

10.56 

14.44 

15.56 

16.11 

13.50 

15.70 

19.33 

21.00 

MPP 

10.00 

8.00 

6.67 

6.78 

8.17 

7.33 

5.67 

5.67 

SS 

8.56 

8.33 

7.78 

10.56 

11.67 

11.67 

12.50 

12.50 

SE 

9.67 

11.11 

11.67 

11.67 

11.67 

12.50 

9.17 

8.33 

TD 

12.00 

14.44 

13.89 

15.56 

8.67 

9.50 

15.33 

15.33 

TTS 

8.11 

7.00 

5.56 

6.22 

8.33 

7.50 

6.67 

5.83 

CRS 

8.33 

7.78 

6.67 

6.89 

7.33 

6.50 

4.83 

5.67 

FAC 

11.11 

8.89 

10.00 

9.11 

8.67 

8.67 

7.83 

5.33 

PHST 

9.44 

7.78 

7.78 

7.44 

12.33 

11.50 

9.83 

11.50 

DI 

12.22 

12.22 

13.33 

9.67 

9.67 

9.67 

8.83 

8.83 
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APPENDIX  D 


Relative  Importance  of  the  ILS  Elements  in  the 
Acquisition  Phases 

In  this  appendix  the  following  abbreviations  are  used: 

MP  Maintenance  Planning 

NPP  Manpower  and  Personnel 

SS  Supply  Support 

SE  Support  Equipment 

TD  Technical  Data 

TTS  Training  and  Training  Support 

CRS  Computer  Resources  Support 

FAC  Facilities 

PHST  Packaging,  Handling,  Storage,  and 

Transportation 
DI  Design  Interface 

Fa  Designates  the  acquisition  logisticians  for 

the  five  program  management  orientations. 

Fb  Designates  the  support  logisticians  for  the 

five  program  management  orientations. 

Faxb  Designates  the  interaction  between  the 

support  logisticians  and  the  acquisition 
logisticians . 

F  Obs  The  observed  value  of  the  F  test. 

F  .01  The  critical  value  of  F  a‘t  the  .01  alpha 


level . 


Results  of  the  Analysis  of  Variance  in  the  Concept 

Exploration  Phase 


ILS 

ELEMENT 

F  Obs 

F.Ol 

F  Obs 

F.Ol 

F 

Obs 

MP 

Fa 

3.26 

3.48 

Fb 

.44 

3.48 

Faxb 

.39 

MPP 

Fa 

.97 

3.48 

Fb 

.09 

3.48 

Faxb 

.18 

SS 

Fa 

1.38 

3.48 

Fb 

.62 

3.48 

Faxb 

.11 

SE 

Fa 

.61 

3.48 

Fb 

.09 

3.48 

Faxb 

.28 

TD 

Fa 

1.62 

3.48 

Fb 

.33 

3.48 

Faxb 

.21 

TTS 

Fa 

.98 

3.48 

Fb 

1.68 

3.48 

Faxb 

.24 

CRS 

Fa 

.05 

3.48 

Fb 

.01 

3.48 

Faxb 

.03 

FAC 

Fa 

2.78 

3.48 

Fb 

.02 

3.48 

Faxb 

.12 

PHST 

Fa 

3.37 

3.48 

Fb 

2.38 

3.48 

Faxb 

.01 

DI 

Fa 

2.55 

3.48 

Fb 

1.22 

3.48 

Faxb 

.03 

Results  of  the  Analysis  of  Variance  in  the 
Demonstration/Validation  Phase 


ILS 


ELEMENT 

F  Obs 

F.Ol 

F  Obs 

F.Ol 

F 

Obs 

MP 

Fa 

.69 

3.48 

Fb 

.02 

3.48 

Faxb 

.55 

MPP 

Fa 

1.96 

3.48 

Fb 

.25 

3.48 

Faxb 

.07 

SS 

Fa 

1.01 

3.48 

Fb 

1.10 

3.48 

Faxb 

.26 

SE 

Fa 

.54 

3.48 

Fb 

.21 

3.48 

Faxb 

.32 

TD 

Pa 

1.63 

3.48 

Fb 

.08 

3.43 

Faxb 

.28 

TTS 

Fa 

.26 

3.48 

Fb 

.11 

3.48 

Faxb 

.02 

CRS 

Fa 

1.06 

3.48 

Fb 

.01 

3.48 

Faxb 

.24 

FAC 

Fa 

1.40 

3.48 

Fb 

.10 

3.48 

Faxb 

.26 

PHST 

Fa 

2.87 

3.48 

Fb 

.67 

3.48 

Faxb 

.17 

DI 

Pa 

1.40 

3.48 

Fb 

.86 

3.48 

Faxb 

.06 

F.Ol 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 


F.Ol 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 

1.89 
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Results  of  the  Analysis  of  Variance  in  the  Full  Scale 

Development  Phase 


ILS 

ELEMENT 

F  Obs 

F.Ol 

F  Obs 

F.Ol 

F 

Obs 

F.Ol 

HP 

Fa 

1.43 

3.48 

Fb 

1.17 

3.48 

Faxb 

.27 

1.89 

MPP 

Fa 

1.54 

3.48 

Fb 

.46 

3.48 

Faxb 

.06 

1.89 

SS 

Fa 

.77 

3.48 

Fb 

.02 

3.48 

Faxb 

.36 

1.89 

SE 

Fa 

.75 

3.48 

Fb 

.32 

3.48 

Faxb 

.32 

1.89 

TD 

Fa 

1.97 

3.48 

Fb 

.03 

3.48 

Faxb 

.04 

1.89 

TTS 

Fa 

1.10 

3.48 

Fb 

.79 

3.48 

Faxb 

.27 

1.89 

CRS 

Fa 

2.40 

3.48 

Fb 

.14 

3.48 

Faxb 

.34 

1.89 

FAC 

Fa 

.81 

3.48 

Fb 

.08 

3.48 

Faxb 

.08 

1.89 

PHST 

Fa 

1.47 

3.48 

Fb 

.01 

3.48 

Faxb 

.14 

1.89 

DI 

Fa 

.58 

3.48 

Fb 

.17 

3.48 

Faxb 

.18 

1.89 

Results  of  the  Analysis  of  Variance  in  the  Production 

Deployment  Phase 


ILS 

ELEMENT 

F  Obs 

F.Ol 

F  Obs 

F.Ol 

F 

Obs 

F.Ol 

MP 

Fa 

3.40 

3.48 

Fb 

2.79 

3.48 

Faxb 

.48 

1.89 

MPP 

Fa 

2.77 

3.48 

Fb 

.25 

3.48 

Faxb 

.27 

1.89 

SS 

Fa 

.24 

3.48 

Fb 

.36 

3.48 

Faxb 

.11 

1.89 

SE 

Fa 

.93 

3.48 

Fb 

.01 

3.48 

Faxb 

,56 

1.89 

TD 

Fa 

1.44 

3.48 

Fb 

.01 

3.48 

Faxb 

.29 

1.89 

TTS 

Fa 

3.30 

3.48 

Fb 

2.40 

3.48 

Faxb 

.01 

1.89 

CRS 

Fa 

1.05 

3.48 

Fb 

.21 

3.48 

Faxb 

.17 

1.89 

FAC 

Fa 

.67 

3.48 

Fb 

.01 

3.48 

Faxb 

.30 

1.89 

PHST 

Fa 

.80 

3.48 

Fb 

.13 

3.48 

Faxb 

.32 

1.89 

DI 

Fa 

1.14 

3.48 

Fb 

.63 

3.48 

Faxb 

.11 

1.89 
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Appendix  E 

Relative  Importance  of  the  Phases  to  the  Acquisition  Process 


Concept  Exploration  Phase 


Mean  Weights  Assigned  By 

Logisticians 

Program  Orientation 

Acquisition 

Support 

Aircraft  Systems 

22.21 

23.93 

Electronic  Systems 

26.07 

20.38 

Missile  Systems 

20.50 

22.86 

Munition  Systems 

28.89 

15.83 

Space  Based  Systems 

19.44 

21.67 

Total 

24.01 

23.78 

HO:  There  is  no  difference  between  weights  assigned  by 
Acquisition  Logisticians  and  Support  Logisticians  based  on 
program  orientation. 


F  OBs 

F.Ol 

Fa 

4.33 

3.48 

Fb 

7.73 

3.48 

Faxb 

1.06 

1.89 

The  observed  value  of  F  exceeds  the  F.Ol  value  for  Fa  and 
Fb.  Therefore,  we  can  reject  the  HO  of  no  difference 
between  the  weights  based  on  program  orientation. 

The  difference  in  these  means  appears  to  be  due  to  the  small 
size  of  the  sample  for  three  of  the  groups. 


Demonstration/Validation  Phase 
Mean  Weights  Assigned  By  Logisticians 


Program  Orientation 

Acquisition 

Support 

Aircraft  Systems 

29.34 

28.47 

Electronic  Systems 

31.43 

24.62 

Missile  Systems 

23.00 

25.00 

Munition  Systems 

25.56 

24.17 

Space  Based  Systems 

22.78 

25.00 

Total 

27.75 

25.89 

HO:  There  is  no  difference  between  weights  assigned  by 

Acquisition  Logisticians  and  Support  Logisticians  based 

program  orientation. 

F  OBs 

F.Ol 

Fa 

3.27 

3.48 

Fb 

.50 

3.48 

Faxb 

.32 

1.89 

The  observed  value  of 

F  does  not  exceed 

the  F.Ol  value 

any  of  the  F  values. 

Therefore,  we  can 

not  reject  the  l 

no  difference  between  the  weights. 
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Full  Scale  Development  Phase 
Mean  Weights  Assigned  By  Logisticians 


Program  Orientation 

Acquisition 

Support 

Aircraft  Systems 

28.50 

26.13 

Electronic  Systems 

25.36 

28.85 

Missile  Systems 

33.50 

27.14 

Munition  Systems 

25.00 

28.33 

Space  Based  Systems 

28.33 

25.00 

Total 

28.16 

27.17 

HO:  There  is  no  difference  between  weights  assigned  by 

Acquisition  Logisticians  and  Support 

program  orientation. 

F  OBS 

Logisticians  based 

F.Ol 

Fa 

3.27 

3.48 

Fb 

.10 

3.48 

Faxb 

.30 

1.89 

The  observed  value  of  F  does  not  exceed  the  F.Ol  value  for 
any  of  the  F  values.  Therefore,  we  can  not  reject  the  HO  of 
no  difference  between  the  weights. 


91 


> 

Production/Deployment  Phase 
Mean  Weights  Assigned  By  Logisticians 


Program  Orientation 

Acquisition 

Support 

Aircraft  Systems 

19.89 

21.47 

Electronic  Systems 

17.14 

26.15 

Missile  Systems 

23.00 

25.00 

Munition  Systems 

20.56 

31.67 

Space  Based  Systems 

29.44 

28.33 

Total 

20.95 

25.47 

HO:  There  is  no  difference  between  weights  assigned  by 

Acquisition  Logisticians  and  Support 

program  orientation. 

F  OBs 

Logisticians  based 

F.Ol 

Fa 

2.79 

3.48 

Fb 

1.63 

3.48 

Faxb 

.29 

1.89 

The  observed  value  of  F  does  not  exceed  the  F.Ol  value  for 
any  of  the  F  values.  Therefore,  we  can  not  reject  the  HO  of 
no  difference  between  the  weights. 
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Appendix  P 

Rank-Order  o£  the  ILS  Elements 


Acquisition  Logisticians 


ILS  ELEMENT 

CE 

DV 

PSD 

PRO 

Maintenance  Planning 

16.21 

14.62 

12.96 

10.98 

Manpower  and  Personnel 

8.90 

8.36 

8.08 

8.36 

Supply  Support 

7.95 

8.45 

10.59 

12.71 

Support  Equipment 

10.44 

10.66 

11.56 

11.55 

Technical  Data 

8.63 

10.38 

11.80 

13.05 

Training  and  Training  Support 

6.41 

6.56 

7.88 

9.86 

Computer  Resources  Support 

7.89 

7.90 

8.06 

8.10 

Facilities 

7.90 

8.95 

8.50 

8.63 

Packaging,  Handling,  Storage, 

and  Transportation 

5.63 

6.20 

7.03 

8.16 

Design  Interface 

20.23 

17.87 

13.55 

8.50 

Pr  obs  -  1.32 
Pr  .01  -  11.3449' 


HO;  There  is  no  difference  between  the  rank  order  of  the  ILS 
elements  based  on  program  phase. 

The  observed  value  of  Pr  does  not  exceed  the  Fr.Ol 
value.  Therefore,  we  can  not  reject  the  HO  of  no  difference 
between  the  rank  order  of  the  elements. 
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Rank-Order  of  the  ILS  Elements 


Support  Logisticians 


ILS  ELEMENT 

CE 

DV 

FSD 

PRO 

Maintenance  Planning 

12.92 

14.96 

15.81 

16.49 

Manpower  and  Personnel 

8.45 

7.64 

7.00 

7.47 

Supply  Support 

9.29 

10.15 

10.32 

11.23 

Support  Equipment 

10.17 

11.51 

10.49 

11.45 

Technical  Data 

9.72 

10.89 

12.17 

13.02 

Training  and  Training  Support 

8.51 

7.02 

6.74 

6.53 

Computer  Resources  Support 

8.02 

8.01 

7.47 

6.53 

Facilities 

8.21 

8.32 

8.09 

8.49 

Packaging,  Handling,  Storage, 

and  Transportation 

8.21 

7.40 

7.02 

7.55 

Design  Interface 

15.53 

14.49 

14.89 

11.23 

Fr  obs  -  2.28 
Fr  .01  -  11.3449 


HO:  There  is  no  difference  between  the  rank  order  of  the  ILS 
elements  based  on  program  phase. 

The  observed  value  of  Fr  does  not  exceed  the  Fr.Ol 
value.  Therefore,  we  can  not  reject  the  HO  of  no  difference 
between  the  rank  order  of  the  elements. 
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